All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC V1 00/12] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-03-28  9:56 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.
   
8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~5 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.
  
* Patch 6 is Kconfig configuration for ISP P1 driver.

* Patch 7 extends the original V4L2 image & meta formats.

* Patch 8 add private v4l2 control ID for ISP P1 driver.

* Patch 9 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.
  
* Patch 10 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
  
* Patch 11 adds communication with the co-processor on the SoC through
  the SCP driver. 
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 12 is current shared memory usage implementation and will be
revised based on Tomasz's suggestion in later version.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

media-ctl -d /dev/media0 -p

Media controller API version 4.19.23

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.23

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "seninf":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: sensor_main (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev5
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "seninf":0 []

- entity 58: sensor_sub (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "seninf":1 []

- entity 60: seninf (12 pads, 3 links)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Sink
		<- "sensor_main":0 []
	pad1: Sink
		<- "sensor_sub":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 []
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

===========
= history =
===========

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.
    
TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to setup media links for
/dev/video1 & /dev/video2.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

Note 3.
Those 24 failures are belonged to same root cause which is caused
by seninf driver. These failures will be fixed in the next
seninf patch set which is mentioned above. Moreover, some OV sensor
issues are already fixed and will be also submitted with seninf
patch set together in the next version.

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media0

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media0:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media0 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media0: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video0:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Su/pported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video1:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video1 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video2:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video2 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video2: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video3:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video3: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video4:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video4 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video4: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video5:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video5 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video5: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video6:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video6 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video6: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000049
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x02000055: from remote pad 0x1000041 of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004b
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : sensor_main
	Function         : Camera Sensor
	Pad 0x01000039   : 0: Source
	  Link 0x02000051: to remote pad 0x100003d of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004d
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003a (58)
	Name             : sensor_sub
	Function         : Camera Sensor
	Pad 0x0100003b   : 0: Source
	  Link 0x02000053: to remote pad 0x100003e of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003c (60)
	Name             : seninf
	Function         : Camera Sensor
	Pad 0x0100003d   : 0: Sink
	  Link 0x02000051: from remote pad 0x1000039 of entity 'sensor_main': Data
	Pad 0x0100003e   : 1: Sink
	  Link 0x02000053: from remote pad 0x100003b of entity 'sensor_sub': Data
	Pad 0x0100003f   : 2: Sink
	Pad 0x01000040   : 3: Sink
	Pad 0x01000041   : 4: Source
	  Link 0x02000055: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x01000042   : 5: Source
	Pad 0x01000043   : 6: Source
	Pad 0x01000044   : 7: Source
	Pad 0x01000045   : 8: Source
	Pad 0x01000046   : 9: Source
	Pad 0x01000047   : 10: Source
	Pad 0x01000048   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 125, Succeeded: 101, Failed: 24, Warnings: 0

Grand Total for mtk-cam device /dev/media0: 668, Succeeded: 644, Failed: 24, Warnings: 0

Jungo Lin (12):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added cam-smem dt-bindings
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory driver

 .../bindings/media/mediatek,cam_smem.txt           |   32 +
 .../devicetree/bindings/media/mediatek,camisp.txt  |   59 +
 .../mediatek,reserve-memory-cam_smem.txt           |   44 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi           |   55 +
 drivers/media/platform/Kconfig                     |    2 +
 drivers/media/platform/mtk-isp/Kconfig             |   21 +
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     |  133 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      |  488 ++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      |  215 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c |  398 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |   25 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 include/uapi/linux/videodev2.h                     |   20 +
 24 files changed, 5414 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
1.9.1



^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC V1 00/12] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-03-28  9:56 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.
   
8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~5 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.
  
* Patch 6 is Kconfig configuration for ISP P1 driver.

* Patch 7 extends the original V4L2 image & meta formats.

* Patch 8 add private v4l2 control ID for ISP P1 driver.

* Patch 9 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.
  
* Patch 10 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
  
* Patch 11 adds communication with the co-processor on the SoC through
  the SCP driver. 
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 12 is current shared memory usage implementation and will be
revised based on Tomasz's suggestion in later version.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

media-ctl -d /dev/media0 -p

Media controller API version 4.19.23

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.23

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "seninf":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: sensor_main (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev5
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "seninf":0 []

- entity 58: sensor_sub (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "seninf":1 []

- entity 60: seninf (12 pads, 3 links)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Sink
		<- "sensor_main":0 []
	pad1: Sink
		<- "sensor_sub":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 []
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

===========
= history =
===========

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.
    
TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to setup media links for
/dev/video1 & /dev/video2.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

Note 3.
Those 24 failures are belonged to same root cause which is caused
by seninf driver. These failures will be fixed in the next
seninf patch set which is mentioned above. Moreover, some OV sensor
issues are already fixed and will be also submitted with seninf
patch set together in the next version.

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media0

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media0:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media0 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media0: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video0:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Su/pported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video1:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video1 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video2:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video2 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video2: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video3:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video3: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video4:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video4 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video4: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video5:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video5 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video5: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video6:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video6 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video6: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000049
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x02000055: from remote pad 0x1000041 of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004b
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : sensor_main
	Function         : Camera Sensor
	Pad 0x01000039   : 0: Source
	  Link 0x02000051: to remote pad 0x100003d of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004d
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003a (58)
	Name             : sensor_sub
	Function         : Camera Sensor
	Pad 0x0100003b   : 0: Source
	  Link 0x02000053: to remote pad 0x100003e of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003c (60)
	Name             : seninf
	Function         : Camera Sensor
	Pad 0x0100003d   : 0: Sink
	  Link 0x02000051: from remote pad 0x1000039 of entity 'sensor_main': Data
	Pad 0x0100003e   : 1: Sink
	  Link 0x02000053: from remote pad 0x100003b of entity 'sensor_sub': Data
	Pad 0x0100003f   : 2: Sink
	Pad 0x01000040   : 3: Sink
	Pad 0x01000041   : 4: Source
	  Link 0x02000055: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x01000042   : 5: Source
	Pad 0x01000043   : 6: Source
	Pad 0x01000044   : 7: Source
	Pad 0x01000045   : 8: Source
	Pad 0x01000046   : 9: Source
	Pad 0x01000047   : 10: Source
	Pad 0x01000048   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 125, Succeeded: 101, Failed: 24, Warnings: 0

Grand Total for mtk-cam device /dev/media0: 668, Succeeded: 644, Failed: 24, Warnings: 0

Jungo Lin (12):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added cam-smem dt-bindings
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory driver

 .../bindings/media/mediatek,cam_smem.txt           |   32 +
 .../devicetree/bindings/media/mediatek,camisp.txt  |   59 +
 .../mediatek,reserve-memory-cam_smem.txt           |   44 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi           |   55 +
 drivers/media/platform/Kconfig                     |    2 +
 drivers/media/platform/mtk-isp/Kconfig             |   21 +
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     |  133 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      |  488 ++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      |  215 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c |  398 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |   25 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 include/uapi/linux/videodev2.h                     |   20 +
 24 files changed, 5414 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
1.9.1

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC V1 00/12] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-03-28  9:56 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.
   
8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~5 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.
  
* Patch 6 is Kconfig configuration for ISP P1 driver.

* Patch 7 extends the original V4L2 image & meta formats.

* Patch 8 add private v4l2 control ID for ISP P1 driver.

* Patch 9 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.
  
* Patch 10 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
  
* Patch 11 adds communication with the co-processor on the SoC through
  the SCP driver. 
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 12 is current shared memory usage implementation and will be
revised based on Tomasz's suggestion in later version.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

media-ctl -d /dev/media0 -p

Media controller API version 4.19.23

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.23

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "seninf":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: sensor_main (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev5
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "seninf":0 []

- entity 58: sensor_sub (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "seninf":1 []

- entity 60: seninf (12 pads, 3 links)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Sink
		<- "sensor_main":0 []
	pad1: Sink
		<- "sensor_sub":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 []
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

===========
= history =
===========

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.
    
TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to setup media links for
/dev/video1 & /dev/video2.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

Note 3.
Those 24 failures are belonged to same root cause which is caused
by seninf driver. These failures will be fixed in the next
seninf patch set which is mentioned above. Moreover, some OV sensor
issues are already fixed and will be also submitted with seninf
patch set together in the next version.

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media0

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media0:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media0 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media0: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video0:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Su/pported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video1:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video1 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video2:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video2 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video2: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video3:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video3: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video4:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video4 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video4: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video5:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video5 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video5: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video6:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.23
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video6 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video6: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x03000049
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x02000055: from remote pad 0x1000041 of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004b
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : sensor_main
	Function         : Camera Sensor
	Pad 0x01000039   : 0: Source
	  Link 0x02000051: to remote pad 0x100003d of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004d
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003a (58)
	Name             : sensor_sub
	Function         : Camera Sensor
	Pad 0x0100003b   : 0: Source
	  Link 0x02000053: to remote pad 0x100003e of entity 'seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.23
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.23
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000003c (60)
	Name             : seninf
	Function         : Camera Sensor
	Pad 0x0100003d   : 0: Sink
	  Link 0x02000051: from remote pad 0x1000039 of entity 'sensor_main': Data
	Pad 0x0100003e   : 1: Sink
	  Link 0x02000053: from remote pad 0x100003b of entity 'sensor_sub': Data
	Pad 0x0100003f   : 2: Sink
	Pad 0x01000040   : 3: Sink
	Pad 0x01000041   : 4: Source
	  Link 0x02000055: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x01000042   : 5: Source
	Pad 0x01000043   : 6: Source
	Pad 0x01000044   : 7: Source
	Pad 0x01000045   : 8: Source
	Pad 0x01000046   : 9: Source
	Pad 0x01000047   : 10: Source
	Pad 0x01000048   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(341): doioctl(node, VIDIOC_SUBDEV_S_FMT, &fmt) != ENOTTY
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 125, Succeeded: 101, Failed: 24, Warnings: 0

Grand Total for mtk-cam device /dev/media0: 668, Succeeded: 644, Failed: 24, Warnings: 0

Jungo Lin (12):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added cam-smem dt-bindings
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory driver

 .../bindings/media/mediatek,cam_smem.txt           |   32 +
 .../devicetree/bindings/media/mediatek,camisp.txt  |   59 +
 .../mediatek,reserve-memory-cam_smem.txt           |   44 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi           |   55 +
 drivers/media/platform/Kconfig                     |    2 +
 drivers/media/platform/mtk-isp/Kconfig             |   21 +
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     |  133 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      |  488 ++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      |  215 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c |  398 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |   25 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 include/uapi/linux/v4l2-controls.h                 |    4 +
 include/uapi/linux/videodev2.h                     |   20 +
 24 files changed, 5414 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
1.9.1



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC V1 01/12] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds the binding for describing the reserved shared memory
used to exchange configuration and tuning data between the
co-processor and Pass 1 (P1) unit of the camera ISP system on
Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam_smem.txt           | 44 ++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
new file mode 100644
index 0000000..05c1bf1
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
@@ -0,0 +1,44 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam_smem"
+
+- reg: required for static allocation (see reserved-memory.txt for
+  the detailed usage)
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x00000400 and 0x100000000 due to the co-processer's
+  addressing limitation
+
+- size: required for dynamic allocation. The unit is bytes.
+
+
+Example:
+
+The following example shows the ISP Pass1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			size = <0 0x1400000>;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x50000000>;
+		};
+	};
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 01/12] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the binding for describing the reserved shared memory
used to exchange configuration and tuning data between the
co-processor and Pass 1 (P1) unit of the camera ISP system on
Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam_smem.txt           | 44 ++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
new file mode 100644
index 0000000..05c1bf1
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
@@ -0,0 +1,44 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam_smem"
+
+- reg: required for static allocation (see reserved-memory.txt for
+  the detailed usage)
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x00000400 and 0x100000000 due to the co-processer's
+  addressing limitation
+
+- size: required for dynamic allocation. The unit is bytes.
+
+
+Example:
+
+The following example shows the ISP Pass1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			size = <0 0x1400000>;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x50000000>;
+		};
+	};
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 01/12] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the binding for describing the reserved shared memory
used to exchange configuration and tuning data between the
co-processor and Pass 1 (P1) unit of the camera ISP system on
Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam_smem.txt           | 44 ++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
new file mode 100644
index 0000000..05c1bf1
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam_smem.txt
@@ -0,0 +1,44 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam_smem"
+
+- reg: required for static allocation (see reserved-memory.txt for
+  the detailed usage)
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x00000400 and 0x100000000 due to the co-processer's
+  addressing limitation
+
+- size: required for dynamic allocation. The unit is bytes.
+
+
+Example:
+
+The following example shows the ISP Pass1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			size = <0 0x1400000>;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x50000000>;
+		};
+	};
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 02/12] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881..c73f7ff 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			no-map;
+			size = <0 0x01400000>; /*20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 02/12] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881..c73f7ff 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			no-map;
+			size = <0 0x01400000>; /*20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 02/12] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881..c73f7ff 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		reserve-memory-cam_smem {
+			compatible = "mediatek,reserve-memory-cam_smem";
+			no-map;
+			size = <0 0x01400000>; /*20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 03/12] dt-bindings: mt8183: Added cam-smem dt-bindings
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds the DT binding documentation for the shared memory
between Pass 1 unit of the camera ISP system and the co-processor
in Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,cam_smem.txt           | 32 ++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
new file mode 100644
index 0000000..d34006a
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
@@ -0,0 +1,32 @@
+Mediatek Camera ISP Pass 1 Shared Memory Device
+
+Mediatek Camera ISP Pass 1 Shared Memory Device is used to manage shared
+memory among CPU, Camera ISP Pass 1 hardware and coprocessor. The Camera
+ISP Pass 1 is a hardware unit for processing image signal from the image
+sensor. Camera ISP Pass 1 is responsible for RAW processing and 3A statistics.
+
+It is associated with a reserved memory region
+(Please see Documentation/devicetree/bindings/reserved-memory/mediatek,
+reserve-memory-cam_smem.txt) and and provides the context to
+allocate memory with dma addresses.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-cam_smem" for MT8183.
+
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+
+- mediatek,larb: must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+
+Example:
+	cam_smem: cam_smem {
+		compatible = "mediatek,mt8183-cam_smem";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+	};
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 03/12] dt-bindings: mt8183: Added cam-smem dt-bindings
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the DT binding documentation for the shared memory
between Pass 1 unit of the camera ISP system and the co-processor
in Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,cam_smem.txt           | 32 ++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
new file mode 100644
index 0000000..d34006a
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
@@ -0,0 +1,32 @@
+Mediatek Camera ISP Pass 1 Shared Memory Device
+
+Mediatek Camera ISP Pass 1 Shared Memory Device is used to manage shared
+memory among CPU, Camera ISP Pass 1 hardware and coprocessor. The Camera
+ISP Pass 1 is a hardware unit for processing image signal from the image
+sensor. Camera ISP Pass 1 is responsible for RAW processing and 3A statistics.
+
+It is associated with a reserved memory region
+(Please see Documentation/devicetree/bindings/reserved-memory/mediatek,
+reserve-memory-cam_smem.txt) and and provides the context to
+allocate memory with dma addresses.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-cam_smem" for MT8183.
+
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+
+- mediatek,larb: must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+
+Example:
+	cam_smem: cam_smem {
+		compatible = "mediatek,mt8183-cam_smem";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+	};
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 03/12] dt-bindings: mt8183: Added cam-smem dt-bindings
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the DT binding documentation for the shared memory
between Pass 1 unit of the camera ISP system and the co-processor
in Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,cam_smem.txt           | 32 ++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,cam_smem.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
new file mode 100644
index 0000000..d34006a
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,cam_smem.txt
@@ -0,0 +1,32 @@
+Mediatek Camera ISP Pass 1 Shared Memory Device
+
+Mediatek Camera ISP Pass 1 Shared Memory Device is used to manage shared
+memory among CPU, Camera ISP Pass 1 hardware and coprocessor. The Camera
+ISP Pass 1 is a hardware unit for processing image signal from the image
+sensor. Camera ISP Pass 1 is responsible for RAW processing and 3A statistics.
+
+It is associated with a reserved memory region
+(Please see Documentation/devicetree/bindings/reserved-memory/mediatek,
+reserve-memory-cam_smem.txt) and and provides the context to
+allocate memory with dma addresses.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-cam_smem" for MT8183.
+
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+
+- mediatek,larb: must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+
+Example:
+	cam_smem: cam_smem {
+		compatible = "mediatek,mt8183-cam_smem";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+	};
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 04/12] dt-bindings: mt8183: Added camera ISP Pass 1
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP effects and writes the
image data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek,camisp.txt  | 59 ++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 0000000..5af1e3c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,59 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects and writes the image data
+to DRAM. Furthermore, Pass 1 unit has the ability to output two different
+resolutions frames at the same time to increase the performance of the
+camera application.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-camisp" for MT8183.
+- reg: must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- mediatek,smi : a phandle to the smi_common node.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP).
+- smem_device : the shared memory device managing the shared memory between
+  Pass 1 unit and the video processor unit.
+
+Example:
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		smem_device = <&cam_smem>;
+	};
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 04/12] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP effects and writes the
image data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek,camisp.txt  | 59 ++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 0000000..5af1e3c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,59 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects and writes the image data
+to DRAM. Furthermore, Pass 1 unit has the ability to output two different
+resolutions frames at the same time to increase the performance of the
+camera application.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-camisp" for MT8183.
+- reg: must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- mediatek,smi : a phandle to the smi_common node.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP).
+- smem_device : the shared memory device managing the shared memory between
+  Pass 1 unit and the video processor unit.
+
+Example:
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		smem_device = <&cam_smem>;
+	};
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 04/12] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP effects and writes the
image data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../devicetree/bindings/media/mediatek,camisp.txt  | 59 ++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 0000000..5af1e3c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,59 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects and writes the image data
+to DRAM. Furthermore, Pass 1 unit has the ability to output two different
+resolutions frames at the same time to increase the performance of the
+camera application.
+
+Required properties:
+- compatible: must be "mediatek,mt8183-camisp" for MT8183.
+- reg: must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- mediatek,smi : a phandle to the smi_common node.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP).
+- smem_device : the shared memory device managing the shared memory between
+  Pass 1 unit and the video processor unit.
+
+Example:
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		smem_device = <&cam_smem>;
+	};
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 05/12] dts: arm64: mt8183: Add ISP Pass 1 nodes
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 42 ++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index c73f7ff..7df39bd 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,47 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		cam_smem: cam_smem {
+			compatible = "mediatek,mt8183-cam_smem";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			smem_device = <&cam_smem>;
+		};
+
 	};
 };
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 05/12] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 42 ++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index c73f7ff..7df39bd 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,47 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		cam_smem: cam_smem {
+			compatible = "mediatek,mt8183-cam_smem";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			smem_device = <&cam_smem>;
+		};
+
 	};
 };
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 05/12] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 42 ++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index c73f7ff..7df39bd 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,47 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		cam_smem: cam_smem {
+			compatible = "mediatek,mt8183-cam_smem";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			smem_device = <&cam_smem>;
+		};
+
 	};
 };
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 06/12] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed1..7be62e0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 0000000..9932563
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 06/12] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga-F7+t8E8rja9g9hUCZPvPmw,
	hans.verkuil-FYB4Gu1CFyUAvxtiuMwx3w,
	laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A
  Cc: Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	holmes.chiou-NuS5LvNUpcJWk0Htik3J/w,
	ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	jungo.lin-NuS5LvNUpcJWk0Htik3J/w,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	christie.yu-NuS5LvNUpcJWk0Htik3J/w,
	seraph.huang-NuS5LvNUpcJWk0Htik3J/w,
	zwisler-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed1..7be62e0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 0000000..9932563
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 06/12] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed1..7be62e0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 0000000..9932563
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 07/12] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220d..b79046d 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 07/12] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga-F7+t8E8rja9g9hUCZPvPmw,
	hans.verkuil-FYB4Gu1CFyUAvxtiuMwx3w,
	laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A
  Cc: Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	holmes.chiou-NuS5LvNUpcJWk0Htik3J/w,
	ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	jungo.lin-NuS5LvNUpcJWk0Htik3J/w,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	christie.yu-NuS5LvNUpcJWk0Htik3J/w,
	seraph.huang-NuS5LvNUpcJWk0Htik3J/w,
	zwisler-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220d..b79046d 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 07/12] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220d..b79046d 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 08/12] media: platform: Add Mediatek ISP P1 private control
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     | 133 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |  32 +++++
 include/uapi/linux/v4l2-controls.h                 |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 0000000..455216a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:(%d*%d)",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_info(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 0000000..74a6538
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2..cbe8f5f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 08/12] media: platform: Add Mediatek ISP P1 private control
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     | 133 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |  32 +++++
 include/uapi/linux/v4l2-controls.h                 |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 0000000..455216a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:(%d*%d)",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_info(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 0000000..74a6538
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2..cbe8f5f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 08/12] media: platform: Add Mediatek ISP P1 private control
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c     | 133 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h     |  32 +++++
 include/uapi/linux/v4l2-controls.h                 |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 0000000..455216a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:(%d*%d)",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_info(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 0000000..74a6538
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2..cbe8f5f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 09/12] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

Implement standard V4L2 video driver that utilizes v4l2
and media frameworks. In this driver, supports one sub-device
and six video devices. Moreover, it also connects with sensor
& senif drivers with camera IO connection via media controller APIs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 7 files changed, 2353 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 0000000..8ddc34b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o mtk_cam-dev-ctx-core.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-drv.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
new file mode 100644
index 0000000..5f3b807
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTX_H__
+#define __MTK_CAM_CTX_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/videobuf2-core.h>
+#include <media/v4l2-subdev.h>
+
+#define MTK_CAM_DEV_NODES			11
+#define MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX	MTK_CAM_DEV_NODES
+
+struct mtk_cam_dev;
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer*
+		buffers[MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/* For v4l2 event data, must smaller than 64 bytes */
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+#endif /*__MTK_CAM_CTX_H__*/
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
new file mode 100644
index 0000000..c17b294
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
@@ -0,0 +1,302 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/dma-mapping.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+#include "mtk_cam-smem.h"
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		return cal_main_stream_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		return cal_packed_out_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+
+	return 0;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+
+	return 0;
+}
+
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd(%d), frame_seq(%d) state(%d)\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = stat->frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 0000000..35e77ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 80 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static struct platform_device *
+mtk_cam_dev_of_find_smem_dev(struct platform_device *pdev)
+{
+	struct device_node *smem_dev_node;
+
+	smem_dev_node = of_parse_phandle(pdev->dev.of_node,
+					 "smem_device", 0);
+	if (!smem_dev_node) {
+		dev_err(&pdev->dev,
+			"failed to find isp smem device for (%s)\n",
+			pdev->name);
+		return NULL;
+	}
+
+	dev_dbg(&pdev->dev, "smem of node found, try to discovery device\n");
+	return of_find_device_by_node(smem_dev_node);
+}
+
+/* Initliaze a mtk_cam_dev representing a completed HW ISP device. */
+static int mtk_cam_dev_init(struct mtk_cam_dev *cam_dev,
+			    struct platform_device *pdev)
+{
+	int ret;
+
+	/* v4l2 sub-device registration */
+	dev_info(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		 MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to create V4L2 devices (%d)\n", ret);
+		goto failed_mem2mem2;
+	}
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev, "v4l2 async init failed\n");
+		goto failed_async;
+	}
+
+	return 0;
+
+failed_mem2mem2:
+failed_async:
+	return ret;
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&cam_dev->mem2mem2_nodes[node].slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	dev_dbg(dev, "%s\n", __func__);
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+
+		dev_dbg(dev, "Check node:%d, queue:%d\n",
+			node, cam_dev->mem2mem2_nodes[node].enabled);
+
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf) {
+			dev_warn(dev, "No available buffer of enabled node %d\n",
+				 node);
+			continue;
+		}
+
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr
+				(&cam_dev->smem_pdev->dev,
+				buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Buf: fd:%d idx:%d state:%d daddr:0x%pK scp_addr:0x%pK",
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			buf->daddr,
+			buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev)
+{
+	struct platform_device *smem_dev;
+
+	smem_dev = mtk_cam_dev_of_find_smem_dev(pdev);
+	if (!smem_dev) {
+		dev_err(&pdev->dev, "failed to find smem_dev\n");
+		return -EINVAL;
+	}
+
+	cam_dev->pdev = pdev;
+	cam_dev->smem_pdev = smem_dev;
+
+	mtk_cam_dev_core_queue_setup(cam_dev, &queues_setting);
+	mtk_cam_dev_init(cam_dev, pdev);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 0000000..57c0261
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/version.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ctx.h"
+
+#define MTK_CAM_DEV_NODE_MAX			MTK_CAM_DEV_NODES
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_MAX_SENSORS_NUM		2
+
+/* Sensor index */
+#define MTK_CAM_MAIN_SENSOR		0
+#define MTK_CAM_SUB_SENSOR		1
+
+/* Input video nodes */
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+/* Output video nodes */
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @enabled: indicate stream on or off
+ * @id: id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @vdev_fmt: the V4L2 format of video device
+ * @vdev_apd: the media pad graph object of video device
+ * @vbq: a videobuf queue of video device
+ * @ctrl_handler: the control handler of video device
+ * @desc: the node attributes of video device
+ * @pending_list: list for pending buffers before enqueuing into driver
+ * @lock: serializes vb2 queue and video device operations.
+ * @slock: protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int enabled;
+	unsigned int id;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_dev_node_desc desc;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @cio_enabled: indicate stream on or off
+ * @sensor: sensor sub-device
+ * @seninf: sensor_if sub-device
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct platform_device *smem_pdev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct media_pad *subdev_pads;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct v4l2_async_notifier notifier;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODE_MAX];
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	unsigned int cio_enabled;
+	unsigned int cio_pad_sink;
+	unsigned int dev_node_num;
+	unsigned int streaming;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 0000000..66738ad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1182 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENSOR_MAIN_PAD_SRC		0
+#define MTK_CAM_SENSOR_SUB_PAD_SRC		0
+#define MTK_CAM_SENSOR_IF_PAD_MAIN_SINK		0
+#define MTK_CAM_SENSOR_IF_PAD_SUB_SINK		1
+#define MTK_CAM_SENSOR_IF_PAD_SRC		4
+
+#define SUBDEV_CIO_NAME				"cam-io"
+#define SUBDEV_SENINF_NAME			"seninf"
+#define SUBDEV_SENSOR_NAME			"sensor"
+#define SUBDEV_SENSOR_MAIN_NAME			"sensor_main"
+#define SUBDEV_SENSOR_SUB_NAME			"sensor_sub"
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_discover_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_graph graph;
+	struct media_entity *entity = &cam_dev->subdev.entity;
+	struct media_device *mdev = entity->graph_obj.mdev;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	int ret;
+
+	mutex_lock(&mdev->graph_mutex);
+	ret = media_graph_walk_init(&graph, mdev);
+	if (ret) {
+		mutex_unlock(&mdev->graph_mutex);
+		return ret;
+	}
+
+	sensor = NULL;
+	sensor_if = NULL;
+
+	media_graph_walk_start(&graph, entity);
+	while ((entity = media_graph_walk_next(&graph))) {
+		dev_dbg(dev, "Graph traversal: entity: %s\n", entity->name);
+
+		if (!strcmp(entity->name, SUBDEV_SENINF_NAME)) {
+			sensor_if = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+
+		if (!strncmp(entity->name, SUBDEV_SENSOR_NAME, 6)) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+	}
+	mutex_unlock(&mdev->graph_mutex);
+	media_graph_walk_cleanup(&graph);
+
+	if (!sensor_if) {
+		dev_err(dev, "Sensor IF has not been connected\n");
+		return -EINVAL;
+	}
+	if (!sensor) {
+		dev_err(dev, "Sensor has not been not connected\n");
+		return -EINVAL;
+	}
+	cam_dev->sensor_if = sensor_if;
+	cam_dev->sensor = sensor;
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	/*
+	 * Get sensor interace and sensor sub device.
+	 * If the call succeeds, sensor if and sensor are filled
+	 * in isp_dev->cio->sensor_if and isp_dev->cio->sensor.
+	 */
+	ret = mtk_cam_v4l2_discover_sensor(cam_dev);
+	if (ret) {
+		dev_err(dev, "no sensor/sensor if connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* seninf must stream on first */
+	dev_dbg(dev, "streamed on sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor-if:%s stream on failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	ret = mtk_isp_streamon(dev);
+	if (ret) {
+		dev_err(dev, "Pass 1 stream on failed:%d\n", ret);
+		goto fail_cam_on;
+	}
+	cam_dev->streaming = true;
+
+	dev_dbg(dev, "streamed on Pass 1\n");
+	mtk_cam_dev_queue_buffers(cam_dev);
+
+	return 0;
+
+fail_cam_on:
+	v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "streamed off sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed off sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor_if:%s stream off failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	mtk_isp_streamoff(dev);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d --> %d\n",
+		pad, remote->index);
+
+	if (pad == cam_dev->cio_pad_sink)
+		cam_dev->cio_enabled = !!(flags & MEDIA_LNK_FL_ENABLED);
+	else
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	dev_dbg(dev, "queue vb2_buf: Node(%s) queue id(%d)\n",
+		node->vdev.name,
+		node->id);
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		dev_dbg(dev, "%s: mtk_cam_dev_queue_buffers\n",
+			node->vdev.name);
+		mtk_cam_dev_queue_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = &cam_dev->smem_pdev->dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -EINVAL;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index > node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODE_MAX + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = kcalloc(num_subdev_pads,
+				       sizeof(*cam_dev->subdev_pads),
+				       GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_media_entity;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize connection IO media info. */
+	cam_dev->cio_pad_sink = num_subdev_pads - 1;
+	cam_dev->subdev_pads[cam_dev->cio_pad_sink].flags =
+		MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_info(dev, "register vdev:%d:%s\n", i, vdev->name);
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_media_entity:
+	kfree(cam_dev->subdev_pads);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK", &cam_dev->media_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+struct sensor_async_subdev {
+	struct v4l2_async_subdev asd;
+};
+
+static struct v4l2_subdev *get_subdev_by_name(struct mtk_cam_dev *cam_dev,
+					      char *name)
+{
+	struct device_node *node;
+	struct v4l2_subdev *sd;
+
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node) {
+			if (!strcmp(node->name, name))
+				return sd;
+		}
+	}
+	return NULL;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sd;
+	struct v4l2_subdev *src_sd, *sink_sd;
+	struct device_node *node;
+	int ret;
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	/* Links among sensors, sensor interface and cio */
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node)
+			sd->entity.name = node->name;
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_MAIN_NAME);
+	sink_sd = get_subdev_by_name(cam_dev, SUBDEV_SENINF_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create:%s-->%s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_MAIN_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_MAIN_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_err(dev, "not found: sensor_main(0x%pK), seninf(%pK)\n",
+			src_sd, sink_sd);
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_SUB_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create: %s --> %s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_SUB_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_SUB_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d:\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_warn(dev, "not found: sensor_sub(0x%pK), seninf(0x%pK)\n",
+			 src_sd, sink_sd);
+	}
+
+	ret = media_create_pad_link(&sink_sd->entity,
+				    MTK_CAM_SENSOR_IF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    cam_dev->cio_pad_sink,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			sink_sd->entity.name, cam_dev->subdev.entity.name, ret);
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_info(dev, "%s bound\n", sd->entity.name);
+	if (!strncmp(&sd->entity.name[9],
+		     SUBDEV_SENINF_NAME,
+		     strlen(SUBDEV_SENINF_NAME)))
+		mtk_cam_dev_complete(notifier);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	dev_dbg(dev, "%s: To be implemented\n", __func__);
+
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints
+		(&cam_dev->pdev->dev, &cam_dev->notifier,
+		 sizeof(struct sensor_async_subdev),
+		 mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 0000000..73b3691
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 09/12] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Implement standard V4L2 video driver that utilizes v4l2
and media frameworks. In this driver, supports one sub-device
and six video devices. Moreover, it also connects with sensor
& senif drivers with camera IO connection via media controller APIs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 7 files changed, 2353 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 0000000..8ddc34b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o mtk_cam-dev-ctx-core.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-drv.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
new file mode 100644
index 0000000..5f3b807
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTX_H__
+#define __MTK_CAM_CTX_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/videobuf2-core.h>
+#include <media/v4l2-subdev.h>
+
+#define MTK_CAM_DEV_NODES			11
+#define MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX	MTK_CAM_DEV_NODES
+
+struct mtk_cam_dev;
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer*
+		buffers[MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/* For v4l2 event data, must smaller than 64 bytes */
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+#endif /*__MTK_CAM_CTX_H__*/
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
new file mode 100644
index 0000000..c17b294
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
@@ -0,0 +1,302 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/dma-mapping.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+#include "mtk_cam-smem.h"
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		return cal_main_stream_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		return cal_packed_out_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+
+	return 0;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+
+	return 0;
+}
+
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd(%d), frame_seq(%d) state(%d)\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = stat->frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 0000000..35e77ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 80 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static struct platform_device *
+mtk_cam_dev_of_find_smem_dev(struct platform_device *pdev)
+{
+	struct device_node *smem_dev_node;
+
+	smem_dev_node = of_parse_phandle(pdev->dev.of_node,
+					 "smem_device", 0);
+	if (!smem_dev_node) {
+		dev_err(&pdev->dev,
+			"failed to find isp smem device for (%s)\n",
+			pdev->name);
+		return NULL;
+	}
+
+	dev_dbg(&pdev->dev, "smem of node found, try to discovery device\n");
+	return of_find_device_by_node(smem_dev_node);
+}
+
+/* Initliaze a mtk_cam_dev representing a completed HW ISP device. */
+static int mtk_cam_dev_init(struct mtk_cam_dev *cam_dev,
+			    struct platform_device *pdev)
+{
+	int ret;
+
+	/* v4l2 sub-device registration */
+	dev_info(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		 MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to create V4L2 devices (%d)\n", ret);
+		goto failed_mem2mem2;
+	}
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev, "v4l2 async init failed\n");
+		goto failed_async;
+	}
+
+	return 0;
+
+failed_mem2mem2:
+failed_async:
+	return ret;
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&cam_dev->mem2mem2_nodes[node].slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	dev_dbg(dev, "%s\n", __func__);
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+
+		dev_dbg(dev, "Check node:%d, queue:%d\n",
+			node, cam_dev->mem2mem2_nodes[node].enabled);
+
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf) {
+			dev_warn(dev, "No available buffer of enabled node %d\n",
+				 node);
+			continue;
+		}
+
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr
+				(&cam_dev->smem_pdev->dev,
+				buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Buf: fd:%d idx:%d state:%d daddr:0x%pK scp_addr:0x%pK",
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			buf->daddr,
+			buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev)
+{
+	struct platform_device *smem_dev;
+
+	smem_dev = mtk_cam_dev_of_find_smem_dev(pdev);
+	if (!smem_dev) {
+		dev_err(&pdev->dev, "failed to find smem_dev\n");
+		return -EINVAL;
+	}
+
+	cam_dev->pdev = pdev;
+	cam_dev->smem_pdev = smem_dev;
+
+	mtk_cam_dev_core_queue_setup(cam_dev, &queues_setting);
+	mtk_cam_dev_init(cam_dev, pdev);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 0000000..57c0261
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/version.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ctx.h"
+
+#define MTK_CAM_DEV_NODE_MAX			MTK_CAM_DEV_NODES
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_MAX_SENSORS_NUM		2
+
+/* Sensor index */
+#define MTK_CAM_MAIN_SENSOR		0
+#define MTK_CAM_SUB_SENSOR		1
+
+/* Input video nodes */
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+/* Output video nodes */
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @enabled: indicate stream on or off
+ * @id: id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @vdev_fmt: the V4L2 format of video device
+ * @vdev_apd: the media pad graph object of video device
+ * @vbq: a videobuf queue of video device
+ * @ctrl_handler: the control handler of video device
+ * @desc: the node attributes of video device
+ * @pending_list: list for pending buffers before enqueuing into driver
+ * @lock: serializes vb2 queue and video device operations.
+ * @slock: protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int enabled;
+	unsigned int id;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_dev_node_desc desc;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @cio_enabled: indicate stream on or off
+ * @sensor: sensor sub-device
+ * @seninf: sensor_if sub-device
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct platform_device *smem_pdev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct media_pad *subdev_pads;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct v4l2_async_notifier notifier;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODE_MAX];
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	unsigned int cio_enabled;
+	unsigned int cio_pad_sink;
+	unsigned int dev_node_num;
+	unsigned int streaming;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 0000000..66738ad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1182 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENSOR_MAIN_PAD_SRC		0
+#define MTK_CAM_SENSOR_SUB_PAD_SRC		0
+#define MTK_CAM_SENSOR_IF_PAD_MAIN_SINK		0
+#define MTK_CAM_SENSOR_IF_PAD_SUB_SINK		1
+#define MTK_CAM_SENSOR_IF_PAD_SRC		4
+
+#define SUBDEV_CIO_NAME				"cam-io"
+#define SUBDEV_SENINF_NAME			"seninf"
+#define SUBDEV_SENSOR_NAME			"sensor"
+#define SUBDEV_SENSOR_MAIN_NAME			"sensor_main"
+#define SUBDEV_SENSOR_SUB_NAME			"sensor_sub"
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_discover_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_graph graph;
+	struct media_entity *entity = &cam_dev->subdev.entity;
+	struct media_device *mdev = entity->graph_obj.mdev;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	int ret;
+
+	mutex_lock(&mdev->graph_mutex);
+	ret = media_graph_walk_init(&graph, mdev);
+	if (ret) {
+		mutex_unlock(&mdev->graph_mutex);
+		return ret;
+	}
+
+	sensor = NULL;
+	sensor_if = NULL;
+
+	media_graph_walk_start(&graph, entity);
+	while ((entity = media_graph_walk_next(&graph))) {
+		dev_dbg(dev, "Graph traversal: entity: %s\n", entity->name);
+
+		if (!strcmp(entity->name, SUBDEV_SENINF_NAME)) {
+			sensor_if = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+
+		if (!strncmp(entity->name, SUBDEV_SENSOR_NAME, 6)) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+	}
+	mutex_unlock(&mdev->graph_mutex);
+	media_graph_walk_cleanup(&graph);
+
+	if (!sensor_if) {
+		dev_err(dev, "Sensor IF has not been connected\n");
+		return -EINVAL;
+	}
+	if (!sensor) {
+		dev_err(dev, "Sensor has not been not connected\n");
+		return -EINVAL;
+	}
+	cam_dev->sensor_if = sensor_if;
+	cam_dev->sensor = sensor;
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	/*
+	 * Get sensor interace and sensor sub device.
+	 * If the call succeeds, sensor if and sensor are filled
+	 * in isp_dev->cio->sensor_if and isp_dev->cio->sensor.
+	 */
+	ret = mtk_cam_v4l2_discover_sensor(cam_dev);
+	if (ret) {
+		dev_err(dev, "no sensor/sensor if connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* seninf must stream on first */
+	dev_dbg(dev, "streamed on sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor-if:%s stream on failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	ret = mtk_isp_streamon(dev);
+	if (ret) {
+		dev_err(dev, "Pass 1 stream on failed:%d\n", ret);
+		goto fail_cam_on;
+	}
+	cam_dev->streaming = true;
+
+	dev_dbg(dev, "streamed on Pass 1\n");
+	mtk_cam_dev_queue_buffers(cam_dev);
+
+	return 0;
+
+fail_cam_on:
+	v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "streamed off sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed off sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor_if:%s stream off failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	mtk_isp_streamoff(dev);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d --> %d\n",
+		pad, remote->index);
+
+	if (pad == cam_dev->cio_pad_sink)
+		cam_dev->cio_enabled = !!(flags & MEDIA_LNK_FL_ENABLED);
+	else
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	dev_dbg(dev, "queue vb2_buf: Node(%s) queue id(%d)\n",
+		node->vdev.name,
+		node->id);
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		dev_dbg(dev, "%s: mtk_cam_dev_queue_buffers\n",
+			node->vdev.name);
+		mtk_cam_dev_queue_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = &cam_dev->smem_pdev->dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -EINVAL;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index > node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODE_MAX + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = kcalloc(num_subdev_pads,
+				       sizeof(*cam_dev->subdev_pads),
+				       GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_media_entity;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize connection IO media info. */
+	cam_dev->cio_pad_sink = num_subdev_pads - 1;
+	cam_dev->subdev_pads[cam_dev->cio_pad_sink].flags =
+		MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_info(dev, "register vdev:%d:%s\n", i, vdev->name);
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_media_entity:
+	kfree(cam_dev->subdev_pads);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK", &cam_dev->media_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+struct sensor_async_subdev {
+	struct v4l2_async_subdev asd;
+};
+
+static struct v4l2_subdev *get_subdev_by_name(struct mtk_cam_dev *cam_dev,
+					      char *name)
+{
+	struct device_node *node;
+	struct v4l2_subdev *sd;
+
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node) {
+			if (!strcmp(node->name, name))
+				return sd;
+		}
+	}
+	return NULL;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sd;
+	struct v4l2_subdev *src_sd, *sink_sd;
+	struct device_node *node;
+	int ret;
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	/* Links among sensors, sensor interface and cio */
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node)
+			sd->entity.name = node->name;
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_MAIN_NAME);
+	sink_sd = get_subdev_by_name(cam_dev, SUBDEV_SENINF_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create:%s-->%s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_MAIN_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_MAIN_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_err(dev, "not found: sensor_main(0x%pK), seninf(%pK)\n",
+			src_sd, sink_sd);
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_SUB_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create: %s --> %s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_SUB_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_SUB_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d:\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_warn(dev, "not found: sensor_sub(0x%pK), seninf(0x%pK)\n",
+			 src_sd, sink_sd);
+	}
+
+	ret = media_create_pad_link(&sink_sd->entity,
+				    MTK_CAM_SENSOR_IF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    cam_dev->cio_pad_sink,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			sink_sd->entity.name, cam_dev->subdev.entity.name, ret);
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_info(dev, "%s bound\n", sd->entity.name);
+	if (!strncmp(&sd->entity.name[9],
+		     SUBDEV_SENINF_NAME,
+		     strlen(SUBDEV_SENINF_NAME)))
+		mtk_cam_dev_complete(notifier);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	dev_dbg(dev, "%s: To be implemented\n", __func__);
+
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints
+		(&cam_dev->pdev->dev, &cam_dev->notifier,
+		 sizeof(struct sensor_async_subdev),
+		 mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 0000000..73b3691
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 09/12] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

Implement standard V4L2 video driver that utilizes v4l2
and media frameworks. In this driver, supports one sub-device
and six video devices. Moreover, it also connects with sensor
& senif drivers with camera IO connection via media controller APIs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/isp_50/cam/Makefile |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h      |  116 ++
 .../mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c      |  302 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c      |  525 +++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h      |  166 +++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c         | 1182 ++++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h         |   43 +
 7 files changed, 2353 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 0000000..8ddc34b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o mtk_cam-dev-ctx-core.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-drv.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
new file mode 100644
index 0000000..5f3b807
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctx.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTX_H__
+#define __MTK_CAM_CTX_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/videobuf2-core.h>
+#include <media/v4l2-subdev.h>
+
+#define MTK_CAM_DEV_NODES			11
+#define MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX	MTK_CAM_DEV_NODES
+
+struct mtk_cam_dev;
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer*
+		buffers[MTK_CAM_DEV_FRAME_BUNDLE_BUFFER_MAX];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/* For v4l2 event data, must smaller than 64 bytes */
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+#endif /*__MTK_CAM_CTX_H__*/
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
new file mode 100644
index 0000000..c17b294
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev-ctx-core.c
@@ -0,0 +1,302 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/dma-mapping.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+#include "mtk_cam-smem.h"
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		return cal_main_stream_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		return cal_packed_out_stride(dev, width, pix_fmt,
+					      one_pixel_mode);
+
+	return 0;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+int mtk_cam_dev_core_queue_setup(struct mtk_cam_dev *cam_dev,
+				 struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+
+	return 0;
+}
+
+int mtk_cam_dev_core_job_finish(struct mtk_cam_dev *cam_dev,
+				struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd(%d), frame_seq(%d) state(%d)\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_queue_event_dev_state(struct mtk_cam_dev *cam_dev,
+				      struct mtk_cam_dev_stat_event_data *stat)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = stat->frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 0000000..35e77ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static  const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 80 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_node_desc
+	capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static struct platform_device *
+mtk_cam_dev_of_find_smem_dev(struct platform_device *pdev)
+{
+	struct device_node *smem_dev_node;
+
+	smem_dev_node = of_parse_phandle(pdev->dev.of_node,
+					 "smem_device", 0);
+	if (!smem_dev_node) {
+		dev_err(&pdev->dev,
+			"failed to find isp smem device for (%s)\n",
+			pdev->name);
+		return NULL;
+	}
+
+	dev_dbg(&pdev->dev, "smem of node found, try to discovery device\n");
+	return of_find_device_by_node(smem_dev_node);
+}
+
+/* Initliaze a mtk_cam_dev representing a completed HW ISP device. */
+static int mtk_cam_dev_init(struct mtk_cam_dev *cam_dev,
+			    struct platform_device *pdev)
+{
+	int ret;
+
+	/* v4l2 sub-device registration */
+	dev_info(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		 MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to create V4L2 devices (%d)\n", ret);
+		goto failed_mem2mem2;
+	}
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev, "v4l2 async init failed\n");
+		goto failed_async;
+	}
+
+	return 0;
+
+failed_mem2mem2:
+failed_async:
+	return ret;
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&cam_dev->mem2mem2_nodes[node].slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&cam_dev->mem2mem2_nodes[node].slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	dev_dbg(dev, "%s\n", __func__);
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+
+		dev_dbg(dev, "Check node:%d, queue:%d\n",
+			node, cam_dev->mem2mem2_nodes[node].enabled);
+
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf) {
+			dev_warn(dev, "No available buffer of enabled node %d\n",
+				 node);
+			continue;
+		}
+
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr
+				(&cam_dev->smem_pdev->dev,
+				buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Buf: fd:%d idx:%d state:%d daddr:0x%pK scp_addr:0x%pK",
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			buf->daddr,
+			buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev)
+{
+	struct platform_device *smem_dev;
+
+	smem_dev = mtk_cam_dev_of_find_smem_dev(pdev);
+	if (!smem_dev) {
+		dev_err(&pdev->dev, "failed to find smem_dev\n");
+		return -EINVAL;
+	}
+
+	cam_dev->pdev = pdev;
+	cam_dev->smem_pdev = smem_dev;
+
+	mtk_cam_dev_core_queue_setup(cam_dev, &queues_setting);
+	mtk_cam_dev_init(cam_dev, pdev);
+
+	return 0;
+}
+
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 0000000..57c0261
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/version.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ctx.h"
+
+#define MTK_CAM_DEV_NODE_MAX			MTK_CAM_DEV_NODES
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_MAX_SENSORS_NUM		2
+
+/* Sensor index */
+#define MTK_CAM_MAIN_SENSOR		0
+#define MTK_CAM_SUB_SENSOR		1
+
+/* Input video nodes */
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+/* Output video nodes */
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @enabled: indicate stream on or off
+ * @id: id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @vdev_fmt: the V4L2 format of video device
+ * @vdev_apd: the media pad graph object of video device
+ * @vbq: a videobuf queue of video device
+ * @ctrl_handler: the control handler of video device
+ * @desc: the node attributes of video device
+ * @pending_list: list for pending buffers before enqueuing into driver
+ * @lock: serializes vb2 queue and video device operations.
+ * @slock: protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int enabled;
+	unsigned int id;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_dev_node_desc desc;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @cio_enabled: indicate stream on or off
+ * @sensor: sensor sub-device
+ * @seninf: sensor_if sub-device
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct platform_device *smem_pdev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct media_pad *subdev_pads;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct v4l2_async_notifier notifier;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODE_MAX];
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	unsigned int cio_enabled;
+	unsigned int cio_pad_sink;
+	unsigned int dev_node_num;
+	unsigned int streaming;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_core_init(struct platform_device *pdev,
+			  struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_core_release(struct platform_device *pdev,
+			     struct mtk_cam_dev *cam_dev);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 0000000..66738ad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1182 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENSOR_MAIN_PAD_SRC		0
+#define MTK_CAM_SENSOR_SUB_PAD_SRC		0
+#define MTK_CAM_SENSOR_IF_PAD_MAIN_SINK		0
+#define MTK_CAM_SENSOR_IF_PAD_SUB_SINK		1
+#define MTK_CAM_SENSOR_IF_PAD_SRC		4
+
+#define SUBDEV_CIO_NAME				"cam-io"
+#define SUBDEV_SENINF_NAME			"seninf"
+#define SUBDEV_SENSOR_NAME			"sensor"
+#define SUBDEV_SENSOR_MAIN_NAME			"sensor_main"
+#define SUBDEV_SENSOR_SUB_NAME			"sensor_sub"
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_discover_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_graph graph;
+	struct media_entity *entity = &cam_dev->subdev.entity;
+	struct media_device *mdev = entity->graph_obj.mdev;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+	struct v4l2_subdev *sensor_if;
+	int ret;
+
+	mutex_lock(&mdev->graph_mutex);
+	ret = media_graph_walk_init(&graph, mdev);
+	if (ret) {
+		mutex_unlock(&mdev->graph_mutex);
+		return ret;
+	}
+
+	sensor = NULL;
+	sensor_if = NULL;
+
+	media_graph_walk_start(&graph, entity);
+	while ((entity = media_graph_walk_next(&graph))) {
+		dev_dbg(dev, "Graph traversal: entity: %s\n", entity->name);
+
+		if (!strcmp(entity->name, SUBDEV_SENINF_NAME)) {
+			sensor_if = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+
+		if (!strncmp(entity->name, SUBDEV_SENSOR_NAME, 6)) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor if entity found: %s\n",
+				entity->name);
+		}
+	}
+	mutex_unlock(&mdev->graph_mutex);
+	media_graph_walk_cleanup(&graph);
+
+	if (!sensor_if) {
+		dev_err(dev, "Sensor IF has not been connected\n");
+		return -EINVAL;
+	}
+	if (!sensor) {
+		dev_err(dev, "Sensor has not been not connected\n");
+		return -EINVAL;
+	}
+	cam_dev->sensor_if = sensor_if;
+	cam_dev->sensor = sensor;
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	/*
+	 * Get sensor interace and sensor sub device.
+	 * If the call succeeds, sensor if and sensor are filled
+	 * in isp_dev->cio->sensor_if and isp_dev->cio->sensor.
+	 */
+	ret = mtk_cam_v4l2_discover_sensor(cam_dev);
+	if (ret) {
+		dev_err(dev, "no sensor/sensor if connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* seninf must stream on first */
+	dev_dbg(dev, "streamed on sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor-if:%s stream on failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	ret = mtk_isp_streamon(dev);
+	if (ret) {
+		dev_err(dev, "Pass 1 stream on failed:%d\n", ret);
+		goto fail_cam_on;
+	}
+	cam_dev->streaming = true;
+
+	dev_dbg(dev, "streamed on Pass 1\n");
+	mtk_cam_dev_queue_buffers(cam_dev);
+
+	return 0;
+
+fail_cam_on:
+	v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "streamed off sensor:%s\n",
+		cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor:%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed off sensor-if:%s\n",
+		cam_dev->sensor_if->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "sensor_if:%s stream off failed:%d\n",
+			cam_dev->sensor_if->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	mtk_isp_streamoff(dev);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->sensor_if, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d --> %d\n",
+		pad, remote->index);
+
+	if (pad == cam_dev->cio_pad_sink)
+		cam_dev->cio_enabled = !!(flags & MEDIA_LNK_FL_ENABLED);
+	else
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	dev_dbg(dev, "queue vb2_buf: Node(%s) queue id(%d)\n",
+		node->vdev.name,
+		node->id);
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		dev_dbg(dev, "%s: mtk_cam_dev_queue_buffers\n",
+			node->vdev.name);
+		mtk_cam_dev_queue_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = &cam_dev->smem_pdev->dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -EINVAL;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index > node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODE_MAX + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = kcalloc(num_subdev_pads,
+				       sizeof(*cam_dev->subdev_pads),
+				       GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_media_entity;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize connection IO media info. */
+	cam_dev->cio_pad_sink = num_subdev_pads - 1;
+	cam_dev->subdev_pads[cam_dev->cio_pad_sink].flags =
+		MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_info(dev, "register vdev:%d:%s\n", i, vdev->name);
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_media_entity:
+	kfree(cam_dev->subdev_pads);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK", &cam_dev->media_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+struct sensor_async_subdev {
+	struct v4l2_async_subdev asd;
+};
+
+static struct v4l2_subdev *get_subdev_by_name(struct mtk_cam_dev *cam_dev,
+					      char *name)
+{
+	struct device_node *node;
+	struct v4l2_subdev *sd;
+
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node) {
+			if (!strcmp(node->name, name))
+				return sd;
+		}
+	}
+	return NULL;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sd;
+	struct v4l2_subdev *src_sd, *sink_sd;
+	struct device_node *node;
+	int ret;
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	/* Links among sensors, sensor interface and cio */
+	list_for_each_entry(sd, &cam_dev->v4l2_dev.subdevs, list) {
+		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+			continue;
+		node = to_of_node(sd->fwnode);
+		if (node)
+			sd->entity.name = node->name;
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_MAIN_NAME);
+	sink_sd = get_subdev_by_name(cam_dev, SUBDEV_SENINF_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create:%s-->%s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_MAIN_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_MAIN_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_err(dev, "not found: sensor_main(0x%pK), seninf(%pK)\n",
+			src_sd, sink_sd);
+	}
+
+	src_sd = get_subdev_by_name(cam_dev, SUBDEV_SENSOR_SUB_NAME);
+	if (src_sd && sink_sd) {
+		dev_dbg(dev, "Link create: %s --> %s\n",
+			src_sd->entity.name, sink_sd->entity.name);
+		ret = media_create_pad_link(&src_sd->entity,
+					    MTK_CAM_SENSOR_SUB_PAD_SRC,
+					    &sink_sd->entity,
+					    MTK_CAM_SENSOR_IF_PAD_SUB_SINK,
+					    0);
+		if (ret)
+			dev_err(dev,
+				"fail to create pad link %s %s, ret:%d:\n",
+				src_sd->entity.name, sink_sd->entity.name,
+				ret);
+	} else {
+		dev_warn(dev, "not found: sensor_sub(0x%pK), seninf(0x%pK)\n",
+			 src_sd, sink_sd);
+	}
+
+	ret = media_create_pad_link(&sink_sd->entity,
+				    MTK_CAM_SENSOR_IF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    cam_dev->cio_pad_sink,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			sink_sd->entity.name, cam_dev->subdev.entity.name, ret);
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_info(dev, "%s bound\n", sd->entity.name);
+	if (!strncmp(&sd->entity.name[9],
+		     SUBDEV_SENINF_NAME,
+		     strlen(SUBDEV_SENINF_NAME)))
+		mtk_cam_dev_complete(notifier);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	dev_dbg(dev, "%s: To be implemented\n", __func__);
+
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints
+		(&cam_dev->pdev->dev, &cam_dev->notifier,
+		 sizeof(struct sensor_async_subdev),
+		 mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 0000000..73b3691
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 10/12] media: platform: Add Mediatek ISP P1 device driver
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 3 files changed, 1533 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 0000000..90736a1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SOF_INT_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_INTER_ST		0x026c
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_CTL_SPARE2		0x0058
+#define REG_HW_FRAME_NUM	0x13b8
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 0000000..935cf7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1098 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-ctx.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dumpdmastat(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd(%d) frame_seq_no(%d)\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_core_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev, int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct platform_device *pdev = p1_dev->pdev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(&pdev->dev,
+			"%s frame_seq_no=%d, isp_composer_work->req_num=%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) is done, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) drop, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+			dev_dbg(dev,
+				"event IRQ(0x%x) DMA(0x%x) is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+		} else if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s notify IRQ (0x%x) DMA status (0x%x) for frame_seq_no: %d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, %d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+	} else {
+		if (irqstatus & SOF_INT_ST)
+			isp_dev->current_frame = hw_frame_num;
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x, raw_int3_err:0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dumpdmastat(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev(%s) irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR: %s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* initialize the v4l2 common part */
+	ret = mtk_cam_dev_core_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_core_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	phandle rproc_phandle;
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+		if (!p1_dev->scp_pdev) {
+			dev_err(dev, "Failed to get scp device\n");
+			return -EINVAL;
+		}
+		ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+					   &rproc_phandle);
+		if (ret) {
+			dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+			return ret;
+		}
+
+		p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+		dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+			p1_dev->rproc_handle);
+		if (!p1_dev->rproc_handle) {
+			dev_err(dev, "fail to get rproc_handle\n");
+			return -EINVAL;
+		}
+
+		ret = rproc_boot(p1_dev->rproc_handle);
+		if (ret < 0) {
+			/*
+			 * Return 0 if downloading firmware successfully,
+			 * otherwise it is failed
+			 */
+			dev_err(dev, "rproc_boot failed:%d\n", ret);
+			return -EINVAL;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			return ret;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			return ret;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount = %d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_streamon(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma, idx, i;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(dev, "%s enable_dma:0x%x", __func__, enable_dma);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor(%s) g_fmt on failed(%d)\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	/* Stream on */
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_streamoff(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_stream(isp_ctx, 0);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address pa:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.scp_addr,
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.scp_addr,
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd(0x%x) frame_seq_no(%d) is queued cnt:(%d)\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = ISP_DEV_NAME,
+		.of_match_table = mtk_isp_of_ids,
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 0000000..6d13058
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,288 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-scp.h"
+
+#define ISP_DEV_NAME		"mtk-cam"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	atomic_t scp_state;
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_streamon - start to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamon(struct device *dev);
+
+/**
+ * mtk_isp_streamoff -  stop to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamoff(struct device *dev);
+
+/**
+ * mtk_isp_enqueue - enqueue a frame bundle to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase);
+
+#endif /*__CAMERA_ISP_H*/
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 10/12] media: platform: Add Mediatek ISP P1 device driver
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 3 files changed, 1533 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 0000000..90736a1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SOF_INT_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_INTER_ST		0x026c
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_CTL_SPARE2		0x0058
+#define REG_HW_FRAME_NUM	0x13b8
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 0000000..935cf7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1098 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-ctx.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dumpdmastat(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd(%d) frame_seq_no(%d)\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_core_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev, int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct platform_device *pdev = p1_dev->pdev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(&pdev->dev,
+			"%s frame_seq_no=%d, isp_composer_work->req_num=%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) is done, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) drop, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+			dev_dbg(dev,
+				"event IRQ(0x%x) DMA(0x%x) is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+		} else if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s notify IRQ (0x%x) DMA status (0x%x) for frame_seq_no: %d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, %d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+	} else {
+		if (irqstatus & SOF_INT_ST)
+			isp_dev->current_frame = hw_frame_num;
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x, raw_int3_err:0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dumpdmastat(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev(%s) irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR: %s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* initialize the v4l2 common part */
+	ret = mtk_cam_dev_core_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_core_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	phandle rproc_phandle;
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+		if (!p1_dev->scp_pdev) {
+			dev_err(dev, "Failed to get scp device\n");
+			return -EINVAL;
+		}
+		ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+					   &rproc_phandle);
+		if (ret) {
+			dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+			return ret;
+		}
+
+		p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+		dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+			p1_dev->rproc_handle);
+		if (!p1_dev->rproc_handle) {
+			dev_err(dev, "fail to get rproc_handle\n");
+			return -EINVAL;
+		}
+
+		ret = rproc_boot(p1_dev->rproc_handle);
+		if (ret < 0) {
+			/*
+			 * Return 0 if downloading firmware successfully,
+			 * otherwise it is failed
+			 */
+			dev_err(dev, "rproc_boot failed:%d\n", ret);
+			return -EINVAL;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			return ret;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			return ret;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount = %d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_streamon(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma, idx, i;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(dev, "%s enable_dma:0x%x", __func__, enable_dma);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor(%s) g_fmt on failed(%d)\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	/* Stream on */
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_streamoff(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_stream(isp_ctx, 0);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address pa:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.scp_addr,
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.scp_addr,
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd(0x%x) frame_seq_no(%d) is queued cnt:(%d)\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = ISP_DEV_NAME,
+		.of_match_table = mtk_isp_of_ids,
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 0000000..6d13058
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,288 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-scp.h"
+
+#define ISP_DEV_NAME		"mtk-cam"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	atomic_t scp_state;
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_streamon - start to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamon(struct device *dev);
+
+/**
+ * mtk_isp_streamoff -  stop to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamoff(struct device *dev);
+
+/**
+ * mtk_isp_enqueue - enqueue a frame bundle to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase);
+
+#endif /*__CAMERA_ISP_H*/
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 10/12] media: platform: Add Mediatek ISP P1 device driver
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-regs.h     |  147 +++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.c    | 1098 ++++++++++++++++++++
 .../media/platform/mtk-isp/isp_50/cam/mtk_cam.h    |  288 +++++
 3 files changed, 1533 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 0000000..90736a1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SOF_INT_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_INTER_ST		0x026c
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_CTL_SPARE2		0x0058
+#define REG_HW_FRAME_NUM	0x13b8
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 0000000..935cf7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1098 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-ctx.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dumpdmastat(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd(%d) frame_seq_no(%d)\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_core_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev, int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct platform_device *pdev = p1_dev->pdev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(&pdev->dev,
+			"%s frame_seq_no=%d, isp_composer_work->req_num=%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) is done, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(&pdev->dev,
+				"frame(frame_seq_no=%d) drop, queue_cnt(%d)\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+			dev_dbg(dev,
+				"event IRQ(0x%x) DMA(0x%x) is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+			mtk_cam_dev_queue_event_dev_state(cam_dev,
+							  &event_data);
+		} else if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s notify IRQ (0x%x) DMA status (0x%x) for frame_seq_no: %d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, %d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+	} else {
+		if (irqstatus & SOF_INT_ST)
+			isp_dev->current_frame = hw_frame_num;
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x, raw_int3_err:0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dumpdmastat(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev(%s) irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR: %s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* initialize the v4l2 common part */
+	ret = mtk_cam_dev_core_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_core_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	phandle rproc_phandle;
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+		if (!p1_dev->scp_pdev) {
+			dev_err(dev, "Failed to get scp device\n");
+			return -EINVAL;
+		}
+		ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+					   &rproc_phandle);
+		if (ret) {
+			dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+			return ret;
+		}
+
+		p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+		dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+			p1_dev->rproc_handle);
+		if (!p1_dev->rproc_handle) {
+			dev_err(dev, "fail to get rproc_handle\n");
+			return -EINVAL;
+		}
+
+		ret = rproc_boot(p1_dev->rproc_handle);
+		if (ret < 0) {
+			/*
+			 * Return 0 if downloading firmware successfully,
+			 * otherwise it is failed
+			 */
+			dev_err(dev, "rproc_boot failed:%d\n", ret);
+			return -EINVAL;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			return ret;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			return ret;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount = %d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_streamon(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma, idx, i;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(dev, "%s enable_dma:0x%x", __func__, enable_dma);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor(%s) g_fmt on failed(%d)\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=(%0dx%0d),stride:%d,xsize:%d,crop=(%0dx%0d)\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	/* Stream on */
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_streamoff(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_stream(isp_ctx, 0);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address pa:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.scp_addr,
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address:0x%x iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.scp_addr,
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd(0x%x) frame_seq_no(%d) is queued cnt:(%d)\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = ISP_DEV_NAME,
+		.of_match_table = mtk_isp_of_ids,
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 0000000..6d13058
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,288 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctx.h"
+#include "mtk_cam-scp.h"
+
+#define ISP_DEV_NAME		"mtk-cam"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	atomic_t scp_state;
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_streamon - start to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamon(struct device *dev);
+
+/**
+ * mtk_isp_streamoff -  stop to output image & meta data.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_streamoff(struct device *dev);
+
+/**
+ * mtk_isp_enqueue - enqueue a frame bundle to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    struct mtk_cam_dev_start_param *frameparamsbase);
+
+#endif /*__CAMERA_ISP_H*/
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 11/12] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional command based
to exchange data and flow control.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      | 488 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      | 215 +++++++++
 2 files changed, 703 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 0000000..be5fd36
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,488 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	phys_addr_t scp_mem_va;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	scp_mem_va = scp_get_reserve_mem_virt(SCP_ISP_MEM_ID);
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "isp scp mem: va:0x%llx, pa:0x%llx sz:0x%x\n",
+		scp_mem_va, isp_ctx->scp_mem_pa, size);
+
+	if (scp_mem_va != 0 && size > 0)
+		memset((void *)scp_mem_va, 0, size);
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *),
+			      GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+fail_pages_alloc:
+	kfree(pages);
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied) < 4 &&
+			 atomic_read(&isp_ctx->composing_frame) < 3) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (atomic_read(&isp_ctx->scp_state) == SCP_STATE_INVALID) {
+			dev_err(dev,
+				"ignore IPI type: %d, SCP state %d!\n",
+				isp_composer_work->type,
+				atomic_read(&isp_ctx->scp_state));
+			kfree(isp_composer_work);
+			continue;
+		}
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg,
+		       &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_SCP_STATE:
+			atomic_set(&isp_ctx->scp_state,
+				   ipi_msg.cmd_data[0]);
+			break;
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied >= ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_BOOTING);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.cmd_data[0], config_param,
+	       sizeof(struct p1_config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.cmd_data[0], &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 0000000..abc34e7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_PORT	8
+#define MAX_META_DMA_NODES	4
+
+/* describes the maximum size of a payload */
+#define MTK_IPI_CMD_SIZE	272
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_SCP_STATE,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	__u64 drv_data;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		__u8 cmd_data[MTK_IPI_CMD_SIZE - sizeof(__u8) - sizeof(__u64)];
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_p1_state {
+	SCP_STATE_INVALID = 0,
+	SCP_STATE_BOOTING,
+	SCP_STATE_RBREADY,
+};
+
+struct isp_scp_p1_param {
+	struct list_head list_entry;
+	struct mtk_isp_scp_p1_cmd cmd;
+};
+
+#endif /* _MTK_ISP_SCP_H */
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 11/12] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga-F7+t8E8rja9g9hUCZPvPmw,
	hans.verkuil-FYB4Gu1CFyUAvxtiuMwx3w,
	laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A
  Cc: Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	holmes.chiou-NuS5LvNUpcJWk0Htik3J/w,
	ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	jungo.lin-NuS5LvNUpcJWk0Htik3J/w,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	christie.yu-NuS5LvNUpcJWk0Htik3J/w,
	seraph.huang-NuS5LvNUpcJWk0Htik3J/w,
	zwisler-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional command based
to exchange data and flow control.

Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      | 488 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      | 215 +++++++++
 2 files changed, 703 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 0000000..be5fd36
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,488 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	phys_addr_t scp_mem_va;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	scp_mem_va = scp_get_reserve_mem_virt(SCP_ISP_MEM_ID);
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "isp scp mem: va:0x%llx, pa:0x%llx sz:0x%x\n",
+		scp_mem_va, isp_ctx->scp_mem_pa, size);
+
+	if (scp_mem_va != 0 && size > 0)
+		memset((void *)scp_mem_va, 0, size);
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *),
+			      GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+fail_pages_alloc:
+	kfree(pages);
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied) < 4 &&
+			 atomic_read(&isp_ctx->composing_frame) < 3) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (atomic_read(&isp_ctx->scp_state) == SCP_STATE_INVALID) {
+			dev_err(dev,
+				"ignore IPI type: %d, SCP state %d!\n",
+				isp_composer_work->type,
+				atomic_read(&isp_ctx->scp_state));
+			kfree(isp_composer_work);
+			continue;
+		}
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg,
+		       &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_SCP_STATE:
+			atomic_set(&isp_ctx->scp_state,
+				   ipi_msg.cmd_data[0]);
+			break;
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied >= ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_BOOTING);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.cmd_data[0], config_param,
+	       sizeof(struct p1_config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.cmd_data[0], &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 0000000..abc34e7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_PORT	8
+#define MAX_META_DMA_NODES	4
+
+/* describes the maximum size of a payload */
+#define MTK_IPI_CMD_SIZE	272
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_SCP_STATE,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	__u64 drv_data;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		__u8 cmd_data[MTK_IPI_CMD_SIZE - sizeof(__u8) - sizeof(__u64)];
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_p1_state {
+	SCP_STATE_INVALID = 0,
+	SCP_STATE_BOOTING,
+	SCP_STATE_RBREADY,
+};
+
+struct isp_scp_p1_param {
+	struct list_head list_entry;
+	struct mtk_isp_scp_p1_cmd cmd;
+};
+
+#endif /* _MTK_ISP_SCP_H */
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 11/12] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional command based
to exchange data and flow control.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c      | 488 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h      | 215 +++++++++
 2 files changed, 703 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 0000000..be5fd36
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,488 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	phys_addr_t scp_mem_va;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	scp_mem_va = scp_get_reserve_mem_virt(SCP_ISP_MEM_ID);
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "isp scp mem: va:0x%llx, pa:0x%llx sz:0x%x\n",
+		scp_mem_va, isp_ctx->scp_mem_pa, size);
+
+	if (scp_mem_va != 0 && size > 0)
+		memset((void *)scp_mem_va, 0, size);
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *),
+			      GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+fail_pages_alloc:
+	kfree(pages);
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied) < 4 &&
+			 atomic_read(&isp_ctx->composing_frame) < 3) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (atomic_read(&isp_ctx->scp_state) == SCP_STATE_INVALID) {
+			dev_err(dev,
+				"ignore IPI type: %d, SCP state %d!\n",
+				isp_composer_work->type,
+				atomic_read(&isp_ctx->scp_state));
+			kfree(isp_composer_work);
+			continue;
+		}
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg,
+		       &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_SCP_STATE:
+			atomic_set(&isp_ctx->scp_state,
+				   ipi_msg.cmd_data[0]);
+			break;
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied >= ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_INVALID);
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_STATE_BOOTING);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.cmd_data[0], config_param,
+	       sizeof(struct p1_config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.cmd_data[0], &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 0000000..abc34e7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_PORT	8
+#define MAX_META_DMA_NODES	4
+
+/* describes the maximum size of a payload */
+#define MTK_IPI_CMD_SIZE	272
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_SCP_STATE,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	__u64 drv_data;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		__u8 cmd_data[MTK_IPI_CMD_SIZE - sizeof(__u8) - sizeof(__u64)];
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_p1_state {
+	SCP_STATE_INVALID = 0,
+	SCP_STATE_BOOTING,
+	SCP_STATE_RBREADY,
+};
+
+struct isp_scp_p1_param {
+	struct list_head list_entry;
+	struct mtk_isp_scp_p1_cmd cmd;
+};
+
+#endif /* _MTK_ISP_SCP_H */
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 12/12] media: platform: Add Mediatek ISP P1 shared memory driver
  2019-03-28  9:56 ` Jungo Lin
  (?)
@ 2019-03-28  9:56   ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

The purpose of this driver is to provide shared memory management
for exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c | 398 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |  25 ++
 2 files changed, 423 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
new file mode 100644
index 0000000..9a92080
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
@@ -0,0 +1,398 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-contiguous.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/iommu.h>
+#include <asm/cacheflush.h>
+
+#define MTK_CAM_SMEM_DEV_NAME "MTK-CAM-SMEM"
+
+struct mtk_cam_smem_drv {
+	struct platform_device *pdev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	int num_smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	unsigned int smem_size;
+};
+
+/*
+ * MTK CAM SMEM DMA ops
+ */
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+static struct reserved_mem *isp_reserved_smem;
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+		struct iommu_domain *smem_dom;
+		dma_addr_t addr;
+		dma_addr_t limit;
+		struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+
+		smem_dom = iommu_get_domain_for_dev(dev);
+		if (!smem_dom) {
+			dev_warn(dev, "No iommu group domain");
+			return 0;
+		}
+
+		addr = iommu_iova_to_phys(smem_dom, iova);
+
+		limit = smem_dev->smem_base + smem_dev->smem_size;
+		if (addr < smem_dev->smem_base || addr >= limit) {
+			dev_err(dev,
+				"Unexpected scp_addr %pa (must >= %pa and <%pa)\n",
+				&addr, &smem_dev->smem_base, &limit);
+			return 0;
+		}
+		dev_dbg(dev, "Pa verifcation pass: %pa(>=%pa, <%pa)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_drv *smem_dev = dev_get_drvdata(dev);
+	int n_pages_align = 0;
+	int size_align = 0;
+	int page_start = 0;
+	unsigned long long offset_p = 0;
+
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+
+	offset_p = (unsigned long long)scp_addr -
+		(unsigned long long)smem_dev->smem_base;
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages_align = size_align >> PAGE_SHIFT;
+	page_start = offset_p >> PAGE_SHIFT;
+	dev_dbg(dev,
+		"%s:page idx:%d,page pa:0x%llx,pa:0x%llx, aligned size:%d pages:%d\n",
+		__func__,
+		page_start,
+		(unsigned long long)page_to_phys(*(smem_dev->smem_pages
+			+ page_start)),
+		(unsigned long long)scp_addr,
+		size_align,
+		n_pages_align
+		);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + page_start,
+		n_pages_align,
+		0, size_align, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_drv *smem_dev,
+				       struct scatterlist *sg)
+{
+	struct device *dev = &smem_dev->pdev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	dma_addr_t addr = (phys_addr_t)sg_phys(sg);
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr 0x%llx from sg\n", addr);
+		return NULL;
+	}
+
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+		dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	flush_cache_vmap((unsigned long long)cpu_addr,
+			 (unsigned long long)cpu_addr + sgl->length);
+	__dma_flush_area(cpu_addr, sgl->length);
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      const struct dma_map_ops *smem_ops)
+{
+	if (!dev->dma_ops)
+		return -EINVAL;
+
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	((struct dma_map_ops *)smem_ops)->get_sgtable =
+		mtk_cam_smem_get_sgtable;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_device =
+		mtk_cam_smem_sync_sg_for_device;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_cpu =
+		mtk_cam_smem_sync_sg_for_cpu;
+	dev->dma_ops = smem_ops;
+
+	return 0;
+}
+
+static const struct dma_map_ops smem_dma_ops = {
+	.get_sgtable = mtk_cam_smem_get_sgtable,
+};
+
+static int mtk_cam_smem_init(struct mtk_cam_smem_drv **mtk_cam_smem_drv_out,
+			     struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *isp_sys;
+	struct device *dev = &pdev->dev;
+
+	isp_sys = devm_kzalloc(dev,
+			       sizeof(*isp_sys), GFP_KERNEL);
+	isp_sys->pdev = pdev;
+	*mtk_cam_smem_drv_out = isp_sys;
+
+	return 0;
+}
+
+static int mtk_cam_smem_drv_probe(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv;
+	int r = 0;
+	struct device *dev = &pdev->dev;
+
+	dev_dbg(dev, "probe mtk_cam_smem_drv\n");
+
+	r = mtk_cam_smem_init(&smem_drv, pdev);
+
+	if (!smem_drv)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, smem_drv);
+
+	if (isp_reserved_smem) {
+		dma_addr_t dma_addr;
+		dma_addr_t addr;
+		struct iommu_domain *smem_dom;
+		unsigned int i;
+		int size_align;
+		struct page **pages;
+		int n_pages;
+		struct sg_table *sgt = &smem_drv->sgt;
+
+		size_align = round_down(isp_reserved_smem->size,
+					PAGE_SIZE);
+		n_pages = size_align >> PAGE_SHIFT;
+
+		pages = kmalloc_array(n_pages, sizeof(struct page *),
+				      GFP_KERNEL);
+
+		if (!pages)
+			return -ENOMEM;
+
+		for (i = 0; i < n_pages; i++)
+			pages[i] = phys_to_page(isp_reserved_smem->base
+						+ i * PAGE_SIZE);
+
+		r = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					      size_align, GFP_KERNEL);
+
+		if (r) {
+			dev_err(dev, "failed to get alloca sg table\n");
+			return -ENOMEM;
+		}
+
+		dma_map_sg_attrs(dev, sgt->sgl, sgt->nents,
+				 DMA_BIDIRECTIONAL,
+				 DMA_ATTR_SKIP_CPU_SYNC);
+
+		dma_addr = sg_dma_address(sgt->sgl);
+		smem_dom = iommu_get_domain_for_dev(dev);
+		addr = iommu_iova_to_phys(smem_dom, dma_addr);
+
+		if (addr != isp_reserved_smem->base)
+			dev_err(dev,
+				"incorrect pa(0x%llx) should be 0x%llx\n",
+			(unsigned long long)addr,
+			(unsigned long long)isp_reserved_smem->base);
+
+		r = dma_declare_coherent_memory(dev,
+						isp_reserved_smem->base,
+			dma_addr, size_align, DMA_MEMORY_EXCLUSIVE);
+
+		dev_dbg(dev,
+			"Coherent mem base(0x%llx,%llx),size(%lx),ret(%d)\n",
+			isp_reserved_smem->base,
+			dma_addr, size_align, r);
+
+		smem_drv->smem_base = isp_reserved_smem->base;
+		smem_drv->smem_size = size_align;
+		smem_drv->smem_pages = pages;
+		smem_drv->num_smem_pages = n_pages;
+		smem_drv->smem_dma_base = dma_addr;
+
+		dev_dbg(dev, "smem_drv setting (0x%llx,%lx,0x%llx,%d)\n",
+			smem_drv->smem_base, smem_drv->smem_size,
+			(unsigned long long)smem_drv->smem_pages,
+			smem_drv->num_smem_pages);
+	}
+
+	r = mtk_cam_smem_setup_dma_ops(dev, &smem_dma_ops);
+
+	return r;
+}
+
+static int mtk_cam_smem_drv_remove(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv =
+		dev_get_drvdata(&pdev->dev);
+
+	kfree(smem_drv->smem_pages);
+	return 0;
+}
+
+static int mtk_cam_smem_drv_suspend(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_resume(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_dummy_cb(struct device *dev)
+{
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_cam_smem_drv_pm_ops = {
+	SET_RUNTIME_PM_OPS(&mtk_cam_smem_drv_dummy_cb,
+			   &mtk_cam_smem_drv_dummy_cb, NULL)
+	SET_SYSTEM_SLEEP_PM_OPS
+		(&mtk_cam_smem_drv_suspend, &mtk_cam_smem_drv_resume)
+};
+
+static const struct of_device_id mtk_cam_smem_drv_of_match[] = {
+	{
+		.compatible = "mediatek,mt8183-cam_smem",
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_cam_smem_drv_of_match);
+
+static struct platform_driver mtk_cam_smem_driver = {
+	.probe = mtk_cam_smem_drv_probe,
+	.remove = mtk_cam_smem_drv_remove,
+	.driver = {
+		.name = MTK_CAM_SMEM_DEV_NAME,
+		.of_match_table =
+			of_match_ptr(mtk_cam_smem_drv_of_match),
+		.pm = &mtk_cam_smem_drv_pm_ops,
+	},
+};
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem
+					 *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: regions without no-map are not yet supported\n");
+		return -EINVAL;
+	}
+
+	isp_reserved_smem = rmem;
+
+	pr_err("Reserved memory: created DMA memory pool at %pa, size %ld MiB\n",
+	       &rmem->base, (unsigned long)rmem->size / SZ_1M);
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam_smem",
+		       mtk_cam_smem_dma_setup);
+
+int __init mtk_cam_smem_drv_init(void)
+{
+	int ret;
+
+	pr_debug("platform_driver_register: mtk_cam_smem_driver\n");
+	ret = platform_driver_register(&mtk_cam_smem_driver);
+	if (ret)
+		pr_warn("isp smem drv init failed, driver didn't probe\n");
+
+	return ret;
+}
+subsys_initcall(mtk_cam_smem_drv_init);
+
+void __exit mtk_cam_smem_drv_ext(void)
+{
+	platform_driver_unregister(&mtk_cam_smem_driver);
+}
+module_exit(mtk_cam_smem_drv_ext);
+MODULE_AUTHOR("Frederic Chen <frederic.chen@mediatek.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek CAM shared memory driver");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 0000000..719ec02
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_SMEM_H__
+#define __MTK_CAM_SMEM_H__
+
+#include <linux/dma-mapping.h>
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif /*__MTK_CAM_SMEM_H__*/
+
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 12/12] media: platform: Add Mediatek ISP P1 shared memory driver
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this driver is to provide shared memory management
for exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c | 398 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |  25 ++
 2 files changed, 423 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
new file mode 100644
index 0000000..9a92080
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
@@ -0,0 +1,398 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-contiguous.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/iommu.h>
+#include <asm/cacheflush.h>
+
+#define MTK_CAM_SMEM_DEV_NAME "MTK-CAM-SMEM"
+
+struct mtk_cam_smem_drv {
+	struct platform_device *pdev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	int num_smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	unsigned int smem_size;
+};
+
+/*
+ * MTK CAM SMEM DMA ops
+ */
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+static struct reserved_mem *isp_reserved_smem;
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+		struct iommu_domain *smem_dom;
+		dma_addr_t addr;
+		dma_addr_t limit;
+		struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+
+		smem_dom = iommu_get_domain_for_dev(dev);
+		if (!smem_dom) {
+			dev_warn(dev, "No iommu group domain");
+			return 0;
+		}
+
+		addr = iommu_iova_to_phys(smem_dom, iova);
+
+		limit = smem_dev->smem_base + smem_dev->smem_size;
+		if (addr < smem_dev->smem_base || addr >= limit) {
+			dev_err(dev,
+				"Unexpected scp_addr %pa (must >= %pa and <%pa)\n",
+				&addr, &smem_dev->smem_base, &limit);
+			return 0;
+		}
+		dev_dbg(dev, "Pa verifcation pass: %pa(>=%pa, <%pa)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_drv *smem_dev = dev_get_drvdata(dev);
+	int n_pages_align = 0;
+	int size_align = 0;
+	int page_start = 0;
+	unsigned long long offset_p = 0;
+
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+
+	offset_p = (unsigned long long)scp_addr -
+		(unsigned long long)smem_dev->smem_base;
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages_align = size_align >> PAGE_SHIFT;
+	page_start = offset_p >> PAGE_SHIFT;
+	dev_dbg(dev,
+		"%s:page idx:%d,page pa:0x%llx,pa:0x%llx, aligned size:%d pages:%d\n",
+		__func__,
+		page_start,
+		(unsigned long long)page_to_phys(*(smem_dev->smem_pages
+			+ page_start)),
+		(unsigned long long)scp_addr,
+		size_align,
+		n_pages_align
+		);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + page_start,
+		n_pages_align,
+		0, size_align, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_drv *smem_dev,
+				       struct scatterlist *sg)
+{
+	struct device *dev = &smem_dev->pdev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	dma_addr_t addr = (phys_addr_t)sg_phys(sg);
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr 0x%llx from sg\n", addr);
+		return NULL;
+	}
+
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+		dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	flush_cache_vmap((unsigned long long)cpu_addr,
+			 (unsigned long long)cpu_addr + sgl->length);
+	__dma_flush_area(cpu_addr, sgl->length);
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      const struct dma_map_ops *smem_ops)
+{
+	if (!dev->dma_ops)
+		return -EINVAL;
+
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	((struct dma_map_ops *)smem_ops)->get_sgtable =
+		mtk_cam_smem_get_sgtable;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_device =
+		mtk_cam_smem_sync_sg_for_device;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_cpu =
+		mtk_cam_smem_sync_sg_for_cpu;
+	dev->dma_ops = smem_ops;
+
+	return 0;
+}
+
+static const struct dma_map_ops smem_dma_ops = {
+	.get_sgtable = mtk_cam_smem_get_sgtable,
+};
+
+static int mtk_cam_smem_init(struct mtk_cam_smem_drv **mtk_cam_smem_drv_out,
+			     struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *isp_sys;
+	struct device *dev = &pdev->dev;
+
+	isp_sys = devm_kzalloc(dev,
+			       sizeof(*isp_sys), GFP_KERNEL);
+	isp_sys->pdev = pdev;
+	*mtk_cam_smem_drv_out = isp_sys;
+
+	return 0;
+}
+
+static int mtk_cam_smem_drv_probe(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv;
+	int r = 0;
+	struct device *dev = &pdev->dev;
+
+	dev_dbg(dev, "probe mtk_cam_smem_drv\n");
+
+	r = mtk_cam_smem_init(&smem_drv, pdev);
+
+	if (!smem_drv)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, smem_drv);
+
+	if (isp_reserved_smem) {
+		dma_addr_t dma_addr;
+		dma_addr_t addr;
+		struct iommu_domain *smem_dom;
+		unsigned int i;
+		int size_align;
+		struct page **pages;
+		int n_pages;
+		struct sg_table *sgt = &smem_drv->sgt;
+
+		size_align = round_down(isp_reserved_smem->size,
+					PAGE_SIZE);
+		n_pages = size_align >> PAGE_SHIFT;
+
+		pages = kmalloc_array(n_pages, sizeof(struct page *),
+				      GFP_KERNEL);
+
+		if (!pages)
+			return -ENOMEM;
+
+		for (i = 0; i < n_pages; i++)
+			pages[i] = phys_to_page(isp_reserved_smem->base
+						+ i * PAGE_SIZE);
+
+		r = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					      size_align, GFP_KERNEL);
+
+		if (r) {
+			dev_err(dev, "failed to get alloca sg table\n");
+			return -ENOMEM;
+		}
+
+		dma_map_sg_attrs(dev, sgt->sgl, sgt->nents,
+				 DMA_BIDIRECTIONAL,
+				 DMA_ATTR_SKIP_CPU_SYNC);
+
+		dma_addr = sg_dma_address(sgt->sgl);
+		smem_dom = iommu_get_domain_for_dev(dev);
+		addr = iommu_iova_to_phys(smem_dom, dma_addr);
+
+		if (addr != isp_reserved_smem->base)
+			dev_err(dev,
+				"incorrect pa(0x%llx) should be 0x%llx\n",
+			(unsigned long long)addr,
+			(unsigned long long)isp_reserved_smem->base);
+
+		r = dma_declare_coherent_memory(dev,
+						isp_reserved_smem->base,
+			dma_addr, size_align, DMA_MEMORY_EXCLUSIVE);
+
+		dev_dbg(dev,
+			"Coherent mem base(0x%llx,%llx),size(%lx),ret(%d)\n",
+			isp_reserved_smem->base,
+			dma_addr, size_align, r);
+
+		smem_drv->smem_base = isp_reserved_smem->base;
+		smem_drv->smem_size = size_align;
+		smem_drv->smem_pages = pages;
+		smem_drv->num_smem_pages = n_pages;
+		smem_drv->smem_dma_base = dma_addr;
+
+		dev_dbg(dev, "smem_drv setting (0x%llx,%lx,0x%llx,%d)\n",
+			smem_drv->smem_base, smem_drv->smem_size,
+			(unsigned long long)smem_drv->smem_pages,
+			smem_drv->num_smem_pages);
+	}
+
+	r = mtk_cam_smem_setup_dma_ops(dev, &smem_dma_ops);
+
+	return r;
+}
+
+static int mtk_cam_smem_drv_remove(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv =
+		dev_get_drvdata(&pdev->dev);
+
+	kfree(smem_drv->smem_pages);
+	return 0;
+}
+
+static int mtk_cam_smem_drv_suspend(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_resume(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_dummy_cb(struct device *dev)
+{
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_cam_smem_drv_pm_ops = {
+	SET_RUNTIME_PM_OPS(&mtk_cam_smem_drv_dummy_cb,
+			   &mtk_cam_smem_drv_dummy_cb, NULL)
+	SET_SYSTEM_SLEEP_PM_OPS
+		(&mtk_cam_smem_drv_suspend, &mtk_cam_smem_drv_resume)
+};
+
+static const struct of_device_id mtk_cam_smem_drv_of_match[] = {
+	{
+		.compatible = "mediatek,mt8183-cam_smem",
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_cam_smem_drv_of_match);
+
+static struct platform_driver mtk_cam_smem_driver = {
+	.probe = mtk_cam_smem_drv_probe,
+	.remove = mtk_cam_smem_drv_remove,
+	.driver = {
+		.name = MTK_CAM_SMEM_DEV_NAME,
+		.of_match_table =
+			of_match_ptr(mtk_cam_smem_drv_of_match),
+		.pm = &mtk_cam_smem_drv_pm_ops,
+	},
+};
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem
+					 *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: regions without no-map are not yet supported\n");
+		return -EINVAL;
+	}
+
+	isp_reserved_smem = rmem;
+
+	pr_err("Reserved memory: created DMA memory pool at %pa, size %ld MiB\n",
+	       &rmem->base, (unsigned long)rmem->size / SZ_1M);
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam_smem",
+		       mtk_cam_smem_dma_setup);
+
+int __init mtk_cam_smem_drv_init(void)
+{
+	int ret;
+
+	pr_debug("platform_driver_register: mtk_cam_smem_driver\n");
+	ret = platform_driver_register(&mtk_cam_smem_driver);
+	if (ret)
+		pr_warn("isp smem drv init failed, driver didn't probe\n");
+
+	return ret;
+}
+subsys_initcall(mtk_cam_smem_drv_init);
+
+void __exit mtk_cam_smem_drv_ext(void)
+{
+	platform_driver_unregister(&mtk_cam_smem_driver);
+}
+module_exit(mtk_cam_smem_drv_ext);
+MODULE_AUTHOR("Frederic Chen <frederic.chen@mediatek.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek CAM shared memory driver");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 0000000..719ec02
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_SMEM_H__
+#define __MTK_CAM_SMEM_H__
+
+#include <linux/dma-mapping.h>
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif /*__MTK_CAM_SMEM_H__*/
+
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC V1 12/12] media: platform: Add Mediatek ISP P1 shared memory driver
@ 2019-03-28  9:56   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-03-28  9:56 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this driver is to provide shared memory management
for exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c | 398 +++++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-smem.h     |  25 ++
 2 files changed, 423 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
new file mode 100644
index 0000000..9a92080
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-drv.c
@@ -0,0 +1,398 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-contiguous.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/iommu.h>
+#include <asm/cacheflush.h>
+
+#define MTK_CAM_SMEM_DEV_NAME "MTK-CAM-SMEM"
+
+struct mtk_cam_smem_drv {
+	struct platform_device *pdev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	int num_smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	unsigned int smem_size;
+};
+
+/*
+ * MTK CAM SMEM DMA ops
+ */
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+static struct reserved_mem *isp_reserved_smem;
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+		struct iommu_domain *smem_dom;
+		dma_addr_t addr;
+		dma_addr_t limit;
+		struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+
+		smem_dom = iommu_get_domain_for_dev(dev);
+		if (!smem_dom) {
+			dev_warn(dev, "No iommu group domain");
+			return 0;
+		}
+
+		addr = iommu_iova_to_phys(smem_dom, iova);
+
+		limit = smem_dev->smem_base + smem_dev->smem_size;
+		if (addr < smem_dev->smem_base || addr >= limit) {
+			dev_err(dev,
+				"Unexpected scp_addr %pa (must >= %pa and <%pa)\n",
+				&addr, &smem_dev->smem_base, &limit);
+			return 0;
+		}
+		dev_dbg(dev, "Pa verifcation pass: %pa(>=%pa, <%pa)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_drv *smem_dev = dev_get_drvdata(dev);
+	int n_pages_align = 0;
+	int size_align = 0;
+	int page_start = 0;
+	unsigned long long offset_p = 0;
+
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+
+	offset_p = (unsigned long long)scp_addr -
+		(unsigned long long)smem_dev->smem_base;
+
+	size_align = round_up(size, PAGE_SIZE);
+	n_pages_align = size_align >> PAGE_SHIFT;
+	page_start = offset_p >> PAGE_SHIFT;
+	dev_dbg(dev,
+		"%s:page idx:%d,page pa:0x%llx,pa:0x%llx, aligned size:%d pages:%d\n",
+		__func__,
+		page_start,
+		(unsigned long long)page_to_phys(*(smem_dev->smem_pages
+			+ page_start)),
+		(unsigned long long)scp_addr,
+		size_align,
+		n_pages_align
+		);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + page_start,
+		n_pages_align,
+		0, size_align, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_drv *smem_dev,
+				       struct scatterlist *sg)
+{
+	struct device *dev = &smem_dev->pdev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	dma_addr_t addr = (phys_addr_t)sg_phys(sg);
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr 0x%llx from sg\n", addr);
+		return NULL;
+	}
+
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+		dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_drv *smem_dev =
+			dev_get_drvdata(dev);
+	void *cpu_addr;
+
+	cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, sgl);
+	flush_cache_vmap((unsigned long long)cpu_addr,
+			 (unsigned long long)cpu_addr + sgl->length);
+	__dma_flush_area(cpu_addr, sgl->length);
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr(0x%llx),vaddr(0x%llx),size(%d),dir(%d)\n",
+		(unsigned long long)sg_phys(sgl),
+		(unsigned long long)cpu_addr,
+		sgl->length,
+		dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      const struct dma_map_ops *smem_ops)
+{
+	if (!dev->dma_ops)
+		return -EINVAL;
+
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	((struct dma_map_ops *)smem_ops)->get_sgtable =
+		mtk_cam_smem_get_sgtable;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_device =
+		mtk_cam_smem_sync_sg_for_device;
+	((struct dma_map_ops *)smem_ops)->sync_sg_for_cpu =
+		mtk_cam_smem_sync_sg_for_cpu;
+	dev->dma_ops = smem_ops;
+
+	return 0;
+}
+
+static const struct dma_map_ops smem_dma_ops = {
+	.get_sgtable = mtk_cam_smem_get_sgtable,
+};
+
+static int mtk_cam_smem_init(struct mtk_cam_smem_drv **mtk_cam_smem_drv_out,
+			     struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *isp_sys;
+	struct device *dev = &pdev->dev;
+
+	isp_sys = devm_kzalloc(dev,
+			       sizeof(*isp_sys), GFP_KERNEL);
+	isp_sys->pdev = pdev;
+	*mtk_cam_smem_drv_out = isp_sys;
+
+	return 0;
+}
+
+static int mtk_cam_smem_drv_probe(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv;
+	int r = 0;
+	struct device *dev = &pdev->dev;
+
+	dev_dbg(dev, "probe mtk_cam_smem_drv\n");
+
+	r = mtk_cam_smem_init(&smem_drv, pdev);
+
+	if (!smem_drv)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, smem_drv);
+
+	if (isp_reserved_smem) {
+		dma_addr_t dma_addr;
+		dma_addr_t addr;
+		struct iommu_domain *smem_dom;
+		unsigned int i;
+		int size_align;
+		struct page **pages;
+		int n_pages;
+		struct sg_table *sgt = &smem_drv->sgt;
+
+		size_align = round_down(isp_reserved_smem->size,
+					PAGE_SIZE);
+		n_pages = size_align >> PAGE_SHIFT;
+
+		pages = kmalloc_array(n_pages, sizeof(struct page *),
+				      GFP_KERNEL);
+
+		if (!pages)
+			return -ENOMEM;
+
+		for (i = 0; i < n_pages; i++)
+			pages[i] = phys_to_page(isp_reserved_smem->base
+						+ i * PAGE_SIZE);
+
+		r = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					      size_align, GFP_KERNEL);
+
+		if (r) {
+			dev_err(dev, "failed to get alloca sg table\n");
+			return -ENOMEM;
+		}
+
+		dma_map_sg_attrs(dev, sgt->sgl, sgt->nents,
+				 DMA_BIDIRECTIONAL,
+				 DMA_ATTR_SKIP_CPU_SYNC);
+
+		dma_addr = sg_dma_address(sgt->sgl);
+		smem_dom = iommu_get_domain_for_dev(dev);
+		addr = iommu_iova_to_phys(smem_dom, dma_addr);
+
+		if (addr != isp_reserved_smem->base)
+			dev_err(dev,
+				"incorrect pa(0x%llx) should be 0x%llx\n",
+			(unsigned long long)addr,
+			(unsigned long long)isp_reserved_smem->base);
+
+		r = dma_declare_coherent_memory(dev,
+						isp_reserved_smem->base,
+			dma_addr, size_align, DMA_MEMORY_EXCLUSIVE);
+
+		dev_dbg(dev,
+			"Coherent mem base(0x%llx,%llx),size(%lx),ret(%d)\n",
+			isp_reserved_smem->base,
+			dma_addr, size_align, r);
+
+		smem_drv->smem_base = isp_reserved_smem->base;
+		smem_drv->smem_size = size_align;
+		smem_drv->smem_pages = pages;
+		smem_drv->num_smem_pages = n_pages;
+		smem_drv->smem_dma_base = dma_addr;
+
+		dev_dbg(dev, "smem_drv setting (0x%llx,%lx,0x%llx,%d)\n",
+			smem_drv->smem_base, smem_drv->smem_size,
+			(unsigned long long)smem_drv->smem_pages,
+			smem_drv->num_smem_pages);
+	}
+
+	r = mtk_cam_smem_setup_dma_ops(dev, &smem_dma_ops);
+
+	return r;
+}
+
+static int mtk_cam_smem_drv_remove(struct platform_device *pdev)
+{
+	struct mtk_cam_smem_drv *smem_drv =
+		dev_get_drvdata(&pdev->dev);
+
+	kfree(smem_drv->smem_pages);
+	return 0;
+}
+
+static int mtk_cam_smem_drv_suspend(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_resume(struct device *dev)
+{
+	return 0;
+}
+
+static int mtk_cam_smem_drv_dummy_cb(struct device *dev)
+{
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_cam_smem_drv_pm_ops = {
+	SET_RUNTIME_PM_OPS(&mtk_cam_smem_drv_dummy_cb,
+			   &mtk_cam_smem_drv_dummy_cb, NULL)
+	SET_SYSTEM_SLEEP_PM_OPS
+		(&mtk_cam_smem_drv_suspend, &mtk_cam_smem_drv_resume)
+};
+
+static const struct of_device_id mtk_cam_smem_drv_of_match[] = {
+	{
+		.compatible = "mediatek,mt8183-cam_smem",
+	},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mtk_cam_smem_drv_of_match);
+
+static struct platform_driver mtk_cam_smem_driver = {
+	.probe = mtk_cam_smem_drv_probe,
+	.remove = mtk_cam_smem_drv_remove,
+	.driver = {
+		.name = MTK_CAM_SMEM_DEV_NAME,
+		.of_match_table =
+			of_match_ptr(mtk_cam_smem_drv_of_match),
+		.pm = &mtk_cam_smem_drv_pm_ops,
+	},
+};
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem
+					 *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: regions without no-map are not yet supported\n");
+		return -EINVAL;
+	}
+
+	isp_reserved_smem = rmem;
+
+	pr_err("Reserved memory: created DMA memory pool at %pa, size %ld MiB\n",
+	       &rmem->base, (unsigned long)rmem->size / SZ_1M);
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam_smem",
+		       mtk_cam_smem_dma_setup);
+
+int __init mtk_cam_smem_drv_init(void)
+{
+	int ret;
+
+	pr_debug("platform_driver_register: mtk_cam_smem_driver\n");
+	ret = platform_driver_register(&mtk_cam_smem_driver);
+	if (ret)
+		pr_warn("isp smem drv init failed, driver didn't probe\n");
+
+	return ret;
+}
+subsys_initcall(mtk_cam_smem_drv_init);
+
+void __exit mtk_cam_smem_drv_ext(void)
+{
+	platform_driver_unregister(&mtk_cam_smem_driver);
+}
+module_exit(mtk_cam_smem_drv_ext);
+MODULE_AUTHOR("Frederic Chen <frederic.chen@mediatek.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mediatek CAM shared memory driver");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 0000000..719ec02
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_SMEM_H__
+#define __MTK_CAM_SMEM_H__
+
+#include <linux/dma-mapping.h>
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif /*__MTK_CAM_SMEM_H__*/
+
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-04-02 10:04   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-02 10:04 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream, Jungo Lin

From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero. Reserved field is filled with zero in media_device_enum_links.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch copies reserved field of media_links_enum from kernel space
to user space.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/media-device.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..f420829 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	if (copy_to_user(ulinks->reserved, &links.reserved,
+			 sizeof(links.reserved)))
+		return -EFAULT;
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-02 10:04   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-02 10:04 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Jungo Lin, Rynn.Wu, srv_heupstream, holmes.chiou,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero. Reserved field is filled with zero in media_device_enum_links.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch copies reserved field of media_links_enum from kernel space
to user space.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/media-device.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..f420829 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	if (copy_to_user(ulinks->reserved, &links.reserved,
+			 sizeof(links.reserved)))
+		return -EFAULT;
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-02 10:04   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-02 10:04 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Jungo Lin, Rynn.Wu, srv_heupstream, holmes.chiou,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero. Reserved field is filled with zero in media_device_enum_links.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch copies reserved field of media_links_enum from kernel space
to user space.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/media-device.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..f420829 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	if (copy_to_user(ulinks->reserved, &links.reserved,
+			 sizeof(links.reserved)))
+		return -EFAULT;
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
  2019-04-02 10:04   ` Jungo Lin
  (?)
@ 2019-04-02 11:33     ` Laurent Pinchart
  -1 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2019-04-02 11:33 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	Sean.Cheng, sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, srv_heupstream, Jungo Lin

Hi Jungo,

Thank you for the patch.

On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> 
> In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> will check whether reserved field of media_links_enum filled
> with zero. Reserved field is filled with zero in media_device_enum_links.
> 
> However, for 32 bit program, the reserved field is missing
> copy from kernel space to user space in media_device_enum_links32
> function.
> 
> This patch copies reserved field of media_links_enum from kernel space
> to user space.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  drivers/media/media-device.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index b8ec886..f420829 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
>  {
>  	struct media_links_enum links;
>  	compat_uptr_t pads_ptr, links_ptr;
> +	int ret;
>  
>  	memset(&links, 0, sizeof(links));
>  
> @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
>  	links.pads = compat_ptr(pads_ptr);
>  	links.links = compat_ptr(links_ptr);
>  
> -	return media_device_enum_links(mdev, &links);
> +	ret = media_device_enum_links(mdev, &links);
> +	if (ret)
> +		return ret;
> +
> +	if (copy_to_user(ulinks->reserved, &links.reserved,
> +			 sizeof(links.reserved)))
> +		return -EFAULT;

I think it would be better to zero the reserved field here instead of
copying it, as we know it has to be zero.

> +
> +	return 0;
>  }
>  
>  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-02 11:33     ` Laurent Pinchart
  0 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2019-04-02 11:33 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	frankie.chiu, hans.verkuil, frederic.chen, seraph.huang,
	linux-media, holmes.chiou, sj.huang, Jungo Lin, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, tfiga, yuzhao, christie.yu, zwisler

Hi Jungo,

Thank you for the patch.

On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> 
> In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> will check whether reserved field of media_links_enum filled
> with zero. Reserved field is filled with zero in media_device_enum_links.
> 
> However, for 32 bit program, the reserved field is missing
> copy from kernel space to user space in media_device_enum_links32
> function.
> 
> This patch copies reserved field of media_links_enum from kernel space
> to user space.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  drivers/media/media-device.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index b8ec886..f420829 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
>  {
>  	struct media_links_enum links;
>  	compat_uptr_t pads_ptr, links_ptr;
> +	int ret;
>  
>  	memset(&links, 0, sizeof(links));
>  
> @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
>  	links.pads = compat_ptr(pads_ptr);
>  	links.links = compat_ptr(links_ptr);
>  
> -	return media_device_enum_links(mdev, &links);
> +	ret = media_device_enum_links(mdev, &links);
> +	if (ret)
> +		return ret;
> +
> +	if (copy_to_user(ulinks->reserved, &links.reserved,
> +			 sizeof(links.reserved)))
> +		return -EFAULT;

I think it would be better to zero the reserved field here instead of
copying it, as we know it has to be zero.

> +
> +	return 0;
>  }
>  
>  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-02 11:33     ` Laurent Pinchart
  0 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2019-04-02 11:33 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	frankie.chiu, hans.verkuil, frederic.chen, seraph.huang,
	linux-media, holmes.chiou, sj.huang, Jungo Lin, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, tfiga, yuzhao, christie.yu, zwisler

Hi Jungo,

Thank you for the patch.

On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> 
> In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> will check whether reserved field of media_links_enum filled
> with zero. Reserved field is filled with zero in media_device_enum_links.
> 
> However, for 32 bit program, the reserved field is missing
> copy from kernel space to user space in media_device_enum_links32
> function.
> 
> This patch copies reserved field of media_links_enum from kernel space
> to user space.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  drivers/media/media-device.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index b8ec886..f420829 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
>  {
>  	struct media_links_enum links;
>  	compat_uptr_t pads_ptr, links_ptr;
> +	int ret;
>  
>  	memset(&links, 0, sizeof(links));
>  
> @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
>  	links.pads = compat_ptr(pads_ptr);
>  	links.links = compat_ptr(links_ptr);
>  
> -	return media_device_enum_links(mdev, &links);
> +	ret = media_device_enum_links(mdev, &links);
> +	if (ret)
> +		return ret;
> +
> +	if (copy_to_user(ulinks->reserved, &links.reserved,
> +			 sizeof(links.reserved)))
> +		return -EFAULT;

I think it would be better to zero the reserved field here instead of
copying it, as we know it has to be zero.

> +
> +	return 0;
>  }
>  
>  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)

-- 
Regards,

Laurent Pinchart

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
  2019-04-02 11:33     ` Laurent Pinchart
  (?)
@ 2019-04-03  0:30       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  0:30 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	Sean.Cheng, sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, srv_heupstream, Jungo Lin

Hi, Laurent,

On Tue, 2019-04-02 at 14:33 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> > From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> > 
> > In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> > will check whether reserved field of media_links_enum filled
> > with zero. Reserved field is filled with zero in media_device_enum_links.
> > 
> > However, for 32 bit program, the reserved field is missing
> > copy from kernel space to user space in media_device_enum_links32
> > function.
> > 
> > This patch copies reserved field of media_links_enum from kernel space
> > to user space.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  drivers/media/media-device.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> > index b8ec886..f420829 100644
> > --- a/drivers/media/media-device.c
> > +++ b/drivers/media/media-device.c
> > @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  {
> >  	struct media_links_enum links;
> >  	compat_uptr_t pads_ptr, links_ptr;
> > +	int ret;
> >  
> >  	memset(&links, 0, sizeof(links));
> >  
> > @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  	links.pads = compat_ptr(pads_ptr);
> >  	links.links = compat_ptr(links_ptr);
> >  
> > -	return media_device_enum_links(mdev, &links);
> > +	ret = media_device_enum_links(mdev, &links);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (copy_to_user(ulinks->reserved, &links.reserved,
> > +			 sizeof(links.reserved)))
> > +		return -EFAULT;
> 
> I think it would be better to zero the reserved field here instead of
> copying it, as we know it has to be zero.
> 

Got it.
We will revise the implementation and deliver the v2 patch.

Thanks,

Jungo 

> > +
> > +	return 0;
> >  }
> >  
> >  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
> 



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-03  0:30       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  0:30 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	frankie.chiu, hans.verkuil, frederic.chen, seraph.huang,
	linux-media, holmes.chiou, sj.huang, Jungo Lin, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, tfiga, yuzhao, christie.yu, zwisler

Hi, Laurent,

On Tue, 2019-04-02 at 14:33 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> > From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> > 
> > In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> > will check whether reserved field of media_links_enum filled
> > with zero. Reserved field is filled with zero in media_device_enum_links.
> > 
> > However, for 32 bit program, the reserved field is missing
> > copy from kernel space to user space in media_device_enum_links32
> > function.
> > 
> > This patch copies reserved field of media_links_enum from kernel space
> > to user space.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  drivers/media/media-device.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> > index b8ec886..f420829 100644
> > --- a/drivers/media/media-device.c
> > +++ b/drivers/media/media-device.c
> > @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  {
> >  	struct media_links_enum links;
> >  	compat_uptr_t pads_ptr, links_ptr;
> > +	int ret;
> >  
> >  	memset(&links, 0, sizeof(links));
> >  
> > @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  	links.pads = compat_ptr(pads_ptr);
> >  	links.links = compat_ptr(links_ptr);
> >  
> > -	return media_device_enum_links(mdev, &links);
> > +	ret = media_device_enum_links(mdev, &links);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (copy_to_user(ulinks->reserved, &links.reserved,
> > +			 sizeof(links.reserved)))
> > +		return -EFAULT;
> 
> I think it would be better to zero the reserved field here instead of
> copying it, as we know it has to be zero.
> 

Got it.
We will revise the implementation and deliver the v2 patch.

Thanks,

Jungo 

> > +
> > +	return 0;
> >  }
> >  
> >  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy
@ 2019-04-03  0:30       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  0:30 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	frankie.chiu, hans.verkuil, frederic.chen, seraph.huang,
	linux-media, holmes.chiou, sj.huang, Jungo Lin, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, tfiga, yuzhao, christie.yu, zwisler

Hi, Laurent,

On Tue, 2019-04-02 at 14:33 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Tue, Apr 02, 2019 at 06:04:04PM +0800, Jungo Lin wrote:
> > From: Jungo Lin <jungo.lin@mediatek.corp-partner.google.com>
> > 
> > In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
> > will check whether reserved field of media_links_enum filled
> > with zero. Reserved field is filled with zero in media_device_enum_links.
> > 
> > However, for 32 bit program, the reserved field is missing
> > copy from kernel space to user space in media_device_enum_links32
> > function.
> > 
> > This patch copies reserved field of media_links_enum from kernel space
> > to user space.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  drivers/media/media-device.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> > index b8ec886..f420829 100644
> > --- a/drivers/media/media-device.c
> > +++ b/drivers/media/media-device.c
> > @@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  {
> >  	struct media_links_enum links;
> >  	compat_uptr_t pads_ptr, links_ptr;
> > +	int ret;
> >  
> >  	memset(&links, 0, sizeof(links));
> >  
> > @@ -513,7 +514,15 @@ static long media_device_enum_links32(struct media_device *mdev,
> >  	links.pads = compat_ptr(pads_ptr);
> >  	links.links = compat_ptr(links_ptr);
> >  
> > -	return media_device_enum_links(mdev, &links);
> > +	ret = media_device_enum_links(mdev, &links);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (copy_to_user(ulinks->reserved, &links.reserved,
> > +			 sizeof(links.reserved)))
> > +		return -EFAULT;
> 
> I think it would be better to zero the reserved field here instead of
> copying it, as we know it has to be zero.
> 

Got it.
We will revise the implementation and deliver the v2 patch.

Thanks,

Jungo 

> > +
> > +	return 0;
> >  }
> >  
> >  #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
> 



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [PATCH] media: media_device_enum_links32: clean a reserved field
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-04-03  1:44   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  1:44 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, Sean.Cheng,
	sj.huang, christie.yu, holmes.chiou, frederic.chen,
	Jerry-ch.Chen, jungo.lin, frankie.chiu, seraph.huang, ryan.yu,
	Rynn.Wu, yuzhao, zwisler, srv_heupstream

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch adds the cleaning a reserved field logic in
media_device_enum_links32 function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes since v1
- Revised the reserved field handling logic

 drivers/media/media-device.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..6893843 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,13 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	memset(ulinks->reserved, 0, sizeof(ulinks->reserved));
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [PATCH] media: media_device_enum_links32: clean a reserved field
@ 2019-04-03  1:44   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  1:44 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch adds the cleaning a reserved field logic in
media_device_enum_links32 function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes since v1
- Revised the reserved field handling logic

 drivers/media/media-device.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..6893843 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,13 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	memset(ulinks->reserved, 0, sizeof(ulinks->reserved));
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [PATCH] media: media_device_enum_links32: clean a reserved field
@ 2019-04-03  1:44   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-04-03  1:44 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: Sean.Cheng, Rynn.Wu, srv_heupstream, holmes.chiou, ryan.yu,
	Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang, yuzhao,
	christie.yu, seraph.huang, zwisler, linux-mediatek,
	frederic.chen, linux-arm-kernel, linux-media

In v4l2-compliance utility, test MEDIA_IOC_ENUM_ENTITIES
will check whether reserved field of media_links_enum filled
with zero.

However, for 32 bit program, the reserved field is missing
copy from kernel space to user space in media_device_enum_links32
function.

This patch adds the cleaning a reserved field logic in
media_device_enum_links32 function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes since v1
- Revised the reserved field handling logic

 drivers/media/media-device.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec886..6893843 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -502,6 +502,7 @@ static long media_device_enum_links32(struct media_device *mdev,
 {
 	struct media_links_enum links;
 	compat_uptr_t pads_ptr, links_ptr;
+	int ret;
 
 	memset(&links, 0, sizeof(links));
 
@@ -513,7 +514,13 @@ static long media_device_enum_links32(struct media_device *mdev,
 	links.pads = compat_ptr(pads_ptr);
 	links.links = compat_ptr(links_ptr);
 
-	return media_device_enum_links(mdev, &links);
+	ret = media_device_enum_links(mdev, &links);
+	if (ret)
+		return ret;
+
+	memset(ulinks->reserved, 0, sizeof(ulinks->reserved));
+
+	return 0;
 }
 
 #define MEDIA_IOC_ENUM_LINKS32		_IOWR('|', 0x02, struct media_links_enum32)
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 00/11] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~4 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.

* Patch 5 is Kconfig configuration for ISP P1 driver.

* Patch 6 extends the original V4L2 image & meta formats.

* Patch 7 add private v4l2 control ID for ISP P1 driver.

* Patch 8 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.

* Patch 9 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.

* Patch 10 adds communication with the co-processor on the SoC through
  the SCP driver.
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 11 provides ISP P1 shard memory management between ISP P1 &
co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

/usr/bin/media-ctl -d /dev/media2 -p

Media controller API version 4.19.36

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.36

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 []
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 []
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED]

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 []

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 []

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video19
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video20
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video21
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		<- "ov5695 2-0036":0 [ENABLED]
	pad1: Sink
		<- "ov2685 4-003c":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 [ENABLED]
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 [ENABLED]

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver & IOMMU driver.
The patches are as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10872547/

[3]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10719217/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to enable media links betwen
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media2

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media2:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media2 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video15 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video16 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video17 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video18 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video19:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video19 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video19: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video20:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video20 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video20: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video21:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video21 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video21: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media2: 668, Succeeded: 668, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------

Jungo Lin (11):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   92 ++
 .../mediatek,reserve-memory-cam-smem.txt      |   42 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   45 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   21 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  133 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  481 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     |  297 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   28 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   20 +
 21 files changed, 5215 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,V2,00/11] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~4 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.

* Patch 5 is Kconfig configuration for ISP P1 driver.

* Patch 6 extends the original V4L2 image & meta formats.

* Patch 7 add private v4l2 control ID for ISP P1 driver.

* Patch 8 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.

* Patch 9 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.

* Patch 10 adds communication with the co-processor on the SoC through
  the SCP driver.
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 11 provides ISP P1 shard memory management between ISP P1 &
co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

/usr/bin/media-ctl -d /dev/media2 -p

Media controller API version 4.19.36

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.36

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 []
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 []
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED]

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 []

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 []

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video19
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video20
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video21
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		<- "ov5695 2-0036":0 [ENABLED]
	pad1: Sink
		<- "ov2685 4-003c":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 [ENABLED]
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 [ENABLED]

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver & IOMMU driver.
The patches are as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10872547/

[3]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10719217/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to enable media links betwen
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media2

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media2:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media2 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video15 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video16 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video17 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video18 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video19:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video19 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video19: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video20:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video20 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video20: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video21:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video21 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video21: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media2: 668, Succeeded: 668, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------

Jungo Lin (11):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   92 ++
 .../mediatek,reserve-memory-cam-smem.txt      |   42 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   45 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   21 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  133 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  481 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     |  297 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   28 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   20 +
 21 files changed, 5215 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0


^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, V2, 00/11] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path. (The
current metadata interface used in meta input and partial meta nodes
is only a temporary solution to kick off the driver development and is
not ready to be reviewed yet):

1. meta input (output video device): connects to ISP P1 sub device. It
accepts the tuning buffer from user.

2. ISP P1 (sub device): connects to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1~4 are dt-bindings & dts information related to
  ISP P1 driver & shared memory.

* Patch 5 is Kconfig configuration for ISP P1 driver.

* Patch 6 extends the original V4L2 image & meta formats.

* Patch 7 add private v4l2 control ID for ISP P1 driver.

* Patch 8 provides V4L2 utility functions & default video devices
  configuration for ISP P1 driver.

* Patch 9 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.

* Patch 10 adds communication with the co-processor on the SoC through
  the SCP driver.
The SCP driver path is listed below:
<URL: https://patchwork.kernel.org/cover/10872547/>

* Patch 11 provides ISP P1 shard memory management between ISP P1 &
co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch:
<URL: https://patchwork.kernel.org/cover/10852957/>

/usr/bin/media-ctl -d /dev/media2 -p

Media controller API version 4.19.36

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.36

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
	pad0: Sink
		<- "MTK-ISP-P1-V4L2 meta input":0 []
	pad1: Source
		-> "MTK-ISP-P1-V4L2 main stream":0 []
	pad2: Source
		-> "MTK-ISP-P1-V4L2 packed out":0 []
	pad3: Source
		-> "MTK-ISP-P1-V4L2 partial meta 0":0 []
	pad4: Source
		-> "MTK-ISP-P1-V4L2 partial meta 1":0 []
	pad5: Source
		-> "MTK-ISP-P1-V4L2 partial meta 2":0 [DYNAMIC]
	pad6: Source
		-> "MTK-ISP-P1-V4L2 partial meta 3":0 [DYNAMIC]
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED]

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
	pad0: Sink
		-> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
	pad0: Source
		<- "MTK-ISP-P1-V4L2":1 []

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
	pad0: Source
		<- "MTK-ISP-P1-V4L2":2 []

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
	pad0: Source
		<- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video19
	pad0: Source
		<- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video20
	pad0: Source
		<- "MTK-ISP-P1-V4L2":5 [DYNAMIC]

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video21
	pad0: Source
		<- "MTK-ISP-P1-V4L2":6 [DYNAMIC]

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		<- "ov5695 2-0036":0 [ENABLED]
	pad1: Sink
		<- "ov2685 4-003c":0 []
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "MTK-ISP-P1-V4L2":11 [ENABLED]
	pad5: Source
	pad6: Source
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Source

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev6
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 [ENABLED]

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver soruces based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middleware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver & IOMMU driver.
The patches are as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10872547/

[3]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10719217/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
Revised testRequests function to bypass V4L2_CTRL_FLAG_READ_ONLY to
avoid VIDIOC_S_EXT_CTRLS failure due to EACASS with read only flag.

[Code]
	if (qctrl.flags & V4L2_CTRL_FLAG_READ_ONLY ||
		qctrl.flags & V4L2_CTRL_FLAG_WRITE_ONLY)
		continue;

Note 2.
Before passing streaming on testing, need to enable media links betwen
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media2

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media2:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media2 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : MTK-ISP-P1-V4L2 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Sink
	  Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video15 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : MTK-ISP-P1-V4L2 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Source
	  Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video16 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : MTK-ISP-P1-V4L2 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Source
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video17 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 2

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : MTK-ISP-P1-V4L2 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Source
	  Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video18 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video19:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : MTK-ISP-P1-V4L2 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Source
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video19 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video19: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video20:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : MTK-ISP-P1-V4L2 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Source
	  Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video20 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video20: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video21:

Driver Info:
	Driver name      : MTK-ISP-P1-V4L2
	Card type        : MTK-ISP-P1-V4L2
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.36
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : MTK-ISP-P1-V4L2 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Source
	  Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data, Dynamic

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video21 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video21: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : MTK-ISP-P1-V4L2
	Function         : Video Statistics
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data, Dynamic
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data, Dynamic
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev4: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam
	Model            : MTK-ISP-P1-V4L2
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.36
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.36
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media2: 668, Succeeded: 668, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------

Jungo Lin (11):
  dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  dts: arm64: mt8183: Add ISP Pass 1 shared memory node
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 private control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   92 ++
 .../mediatek,reserve-memory-cam-smem.txt      |   42 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   45 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   21 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  133 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   32 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  481 +++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     |  297 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   28 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   20 +
 21 files changed, 5215 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, V2, 01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the binding for describing the reserved
shared memory used to exchange ISP configuration and tuning
data between the co-processor and Pass 1 (P1) unit of the
camera ISP system on Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
new file mode 100644
index 000000000000..65a967cff91e
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
@@ -0,0 +1,42 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+For additional details about reserved memory regions see reserved-memory.txt
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam-smem"
+
+- size: required for dynamic allocation. The unit is bytes.
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x40000000 and 0x100000000 due to the co-processer's
+  addressing limitation.
+
+Example:
+
+The following example shows the ISP Pass 1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			size = <0 0x1400000>;
+			no-map;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds the binding for describing the reserved
shared memory used to exchange ISP configuration and tuning
data between the co-processor and Pass 1 (P1) unit of the
camera ISP system on Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
new file mode 100644
index 000000000000..65a967cff91e
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
@@ -0,0 +1,42 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+For additional details about reserved memory regions see reserved-memory.txt
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam-smem"
+
+- size: required for dynamic allocation. The unit is bytes.
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x40000000 and 0x100000000 due to the co-processer's
+  addressing limitation.
+
+Example:
+
+The following example shows the ISP Pass 1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			size = <0 0x1400000>;
+			no-map;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the binding for describing the reserved
shared memory used to exchange ISP configuration and tuning
data between the co-processor and Pass 1 (P1) unit of the
camera ISP system on Mediatek SoCs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

diff --git a/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
new file mode 100644
index 000000000000..65a967cff91e
--- /dev/null
+++ b/Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
@@ -0,0 +1,42 @@
+Mediatek ISP Pass 1 Shared Memory binding
+
+This binding describes the shared memory, which serves the purpose of
+describing the shared memory region used to exchange data between Pass 1
+unit of Image Signal Processor (ISP) and the co-processor in Mediatek
+SoCs.
+
+The co-processor doesn't have the iommu so we need to use the physical
+address to access the shared buffer in the firmware.
+
+The Pass 1 unit of ISP can access memory through the iommu so it
+uses the dma address to access the memory region.
+(See iommu/mediatek,iommu.txt for the detailed description of Mediatek IOMMU)
+
+For additional details about reserved memory regions see reserved-memory.txt
+
+Required properties:
+
+- compatible: must be "mediatek,reserve-memory-cam-smem"
+
+- size: required for dynamic allocation. The unit is bytes.
+
+- alloc-range: required for dynamic allocation. The range must
+  between 0x40000000 and 0x100000000 due to the co-processer's
+  addressing limitation.
+
+Example:
+
+The following example shows the ISP Pass 1 shared memory setup for MT8183.
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			size = <0 0x1400000>;
+			no-map;
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,02/11] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..d5d83a05f8a1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			no-map;
+			size = <0 0x01400000>; /* 20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,02/11] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..d5d83a05f8a1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			no-map;
+			size = <0 0x01400000>; /* 20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,02/11] dts: arm64: mt8183: Add ISP Pass 1 shared memory node
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds a shared memory region used on mt8183 for
exchanging tuning data between co-processor and the
Pass 1 unit of the camera ISP system.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..d5d83a05f8a1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -133,6 +133,19 @@
 		clock-output-names = "clk26m";
 	};
 
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+		cam_mem_reserved: cam_mem_region {
+			compatible = "mediatek,reserve-memory-cam-smem";
+			no-map;
+			size = <0 0x01400000>; /* 20 MB share mem size */
+			alignment = <0 0x1000>;
+			alloc-ranges = <0 0x40000000 0 0x10000000>;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv8-timer";
 		interrupt-parent = <&gic>;
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..759e55a5dfac
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,92 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- memory-region : the reserved shared memory region between Pass 1 unit and
+  system control processor.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		memory-region = <&cam_mem_reserved>;
+	};
+
+Reserved memory specific DT entry (see reserved memory binding for more
+information):
+
+Example:
+SoC specific DT entry:
+
+	cam_mem_reserved: cam_mem_region {
+		compatible = "mediatek,reserve-memory-cam-smem";
+		no-map;
+		size = <0 0x01400000>; / *20 MB share mem size */
+		alignment = <0 0x1000>;
+		alloc-ranges = <0 0x40000000 0 0x10000000>;
+	};
+
+Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
+contain one 'port' child node with child 'endpoint' node. Please refer to
+the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
+and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
+
+Example:
+Board specific DT entry:
+
+	&camisp {
+		port@0 {
+			seninf_0: endpoint {
+				remote-endpoint = <&seninf_core>;
+			};
+		};
+	};
+
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..759e55a5dfac
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,92 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- memory-region : the reserved shared memory region between Pass 1 unit and
+  system control processor.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		memory-region = <&cam_mem_reserved>;
+	};
+
+Reserved memory specific DT entry (see reserved memory binding for more
+information):
+
+Example:
+SoC specific DT entry:
+
+	cam_mem_reserved: cam_mem_region {
+		compatible = "mediatek,reserve-memory-cam-smem";
+		no-map;
+		size = <0 0x01400000>; / *20 MB share mem size */
+		alignment = <0 0x1000>;
+		alloc-ranges = <0 0x40000000 0 0x10000000>;
+	};
+
+Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
+contain one 'port' child node with child 'endpoint' node. Please refer to
+the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
+and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
+
+Example:
+Board specific DT entry:
+
+	&camisp {
+		port@0 {
+			seninf_0: endpoint {
+				remote-endpoint = <&seninf_core>;
+			};
+		};
+	};
+
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..759e55a5dfac
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,92 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Must contain an entry for each entry in reg-names.
+- interrupts: interrupt number to the cpu.
+- iommus: shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- power-domains : a phandle to the power domain of this local arbiter.
+- clocks: device clocks, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
+- mediatek,larb: must contain the local arbiters in the current SOCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : the node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- memory-region : the reserved shared memory region between Pass 1 unit and
+  system control processor.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp", "syscon";
+		reg = <0 0x1a000000 0 0x1000>,
+		      <0 0x1a003000 0 0x1000>,
+		      <0 0x1a004000 0 0x2000>,
+		      <0 0x1a006000 0 0x2000>;
+		reg-names = "camisp",
+		            "cam1",
+		            "cam2",
+		            "cam3";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam1",
+				  "cam2",
+				  "cam3";
+		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+			 <&iommu M4U_PORT_CAM_LSCI1>,
+			 <&iommu M4U_PORT_CAM_BPCI>;
+		#clock-cells = <1>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		/* Camera CCF */
+		clocks = <&camsys CLK_CAM_CAM>,
+			 <&camsys CLK_CAM_CAMTG>;
+		clock-names = "CAMSYS_CAM_CGPDN",
+			      "CAMSYS_CAMTG_CGPDN";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		mediatek,scp = <&scp>;
+		memory-region = <&cam_mem_reserved>;
+	};
+
+Reserved memory specific DT entry (see reserved memory binding for more
+information):
+
+Example:
+SoC specific DT entry:
+
+	cam_mem_reserved: cam_mem_region {
+		compatible = "mediatek,reserve-memory-cam-smem";
+		no-map;
+		size = <0 0x01400000>; / *20 MB share mem size */
+		alignment = <0 0x1000>;
+		alloc-ranges = <0 0x40000000 0 0x10000000>;
+	};
+
+Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
+contain one 'port' child node with child 'endpoint' node. Please refer to
+the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
+and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
+
+Example:
+Board specific DT entry:
+
+	&camisp {
+		port@0 {
+			seninf_0: endpoint {
+				remote-endpoint = <&seninf_core>;
+			};
+		};
+	};
+
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,04/11] dts: arm64: mt8183: Add ISP Pass 1 nodes
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 32 ++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index d5d83a05f8a1..5f195236a762 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,37 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			memory-region = <&cam_mem_reserved>;
+		};
 	};
 };
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,04/11] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 32 ++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index d5d83a05f8a1..5f195236a762 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,37 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			memory-region = <&cam_mem_reserved>;
+		};
 	};
 };
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,04/11] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 32 ++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index d5d83a05f8a1..5f195236a762 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -382,5 +382,37 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "camisp",
+				    "cam1",
+				    "cam2",
+				    "cam3";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam1",
+					  "cam2",
+					  "cam3";
+			iommus = <&iommu M4U_PORT_CAM_LSCI0>,
+				 <&iommu M4U_PORT_CAM_LSCI1>,
+				 <&iommu M4U_PORT_CAM_BPCI>;
+			#clock-cells = <1>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			/* Camera CCF */
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "CAMSYS_CAM_CGPDN",
+				      "CAMSYS_CAMTG_CGPDN";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+			memory-region = <&cam_mem_reserved>;
+		};
 	};
 };
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 05/11] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed189644..7be62e06573b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..9932563d34c1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,05/11] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed189644..7be62e06573b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..9932563d34c1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 05/11] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 4acbed189644..7be62e06573b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..9932563d34c1
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,21 @@
+config VIDEO_MEDIATEK_ISP_PASS1_SUPPORT
+	bool "Mediatek pass 1 image processing function"
+
+	select DMA_SHARED_BUFFER
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_CORE
+	select VIDEOBUF2_V4L2
+	select VIDEOBUF2_MEMOPS
+	select VIDEOBUF2_VMALLOC
+	select MEDIA_CONTROLLER
+
+	default n
+	help
+		Pass 1 driver controls 3A (autofocus, exposure,
+		and white balance) with tuning parameters and outputs
+		the capture image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		capture application such as video recording and still image
+		capture.
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:57   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220da3bcc..b79046d2d812 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220da3bcc..b79046d2d812 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-10  1:57   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:57 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1db220da3bcc..b79046d2d812 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -711,6 +711,20 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP compressed formats */
+#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -732,6 +746,12 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:58   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..520adbe367ed
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..74a6538c81ac
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2fb3ae..cbe8f5f7782b 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..520adbe367ed
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..74a6538c81ac
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2fb3ae..cbe8f5f7782b 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 private control number with 16.
Moreover, add two private controls for ISP P1 user space
usage.

1. V4L2_CID_PRIVATE_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_PRIVATE_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default image path is pure raw.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 3 files changed, 169 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..520adbe367ed
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include "mtk_cam-dev.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+	unsigned int width, height;
+
+	width = imgo_fmt->fmt.pix_mp.width;
+	height = imgo_fmt->fmt.pix_mp.height;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
+		width, height);
+
+	ctrl->val = (width << 16) | height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
+	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_GET_BIN_INFO:
+		handle_ctrl_get_bin_info(ctrl);
+		break;
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		handle_ctrl_get_raw_path(ctrl);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_PRIVATE_RAW_PATH:
+		return handle_ctrl_set_raw_path(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
+	.name = "MTK CAM GET BIN INFO",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
+	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_PRIVATE_RAW_PATH,
+	.name = "MTK CAM RAW PATH",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..74a6538c81ac
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
+#define V4L2_CID_PRIVATE_GET_BIN_INFO \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
+#define V4L2_CID_PRIVATE_RAW_PATH \
+	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
+
+#define V4L2_CID_MTK_CAM_MAX	16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 06479f2fb3ae..cbe8f5f7782b 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:58     ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                       ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga-F7+t8E8rja9g9hUCZPvPmw,
	hans.verkuil-FYB4Gu1CFyUAvxtiuMwx3w,
	laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A
  Cc: shik-F7+t8E8rja9g9hUCZPvPmw, devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	suleiman-F7+t8E8rja9g9hUCZPvPmw, Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w, Jungo Lin,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	seraph.huang-NuS5LvNUpcJWk0Htik3J/w,
	zwisler-F7+t8E8rja9g9hUCZPvPmw,
	christie.yu-NuS5LvNUpcJWk0Htik3J/w,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and six video devices during
initialization. Moreover, it also connects with sensor and
senif drivers with V4L2 async APIs.

Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
---
This patch dependeds on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/
---
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 ++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 5 files changed, 2156 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..5a581ab65945
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 000000000000..dda8a7b161ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,758 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static const struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = cal_main_stream_stride(dev, width, pix_fmt,
+					     one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = cal_packed_out_stride(dev, width, pix_fmt,
+					    one_pixel_mode);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+}
+
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd:%d, frame_seq:%d state:%d\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&vdev->slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&vdev->slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&vdev->slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf)
+			continue;
+
+		/* TBD: use buf_init callback function */
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
+				cam_dev->smem_dev, buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
+			node,
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			&buf->daddr,
+			&buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_req_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+
+	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
+
+	/* v4l2 sub-device registration */
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 000000000000..410460de44fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,250 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_DEV_NODES		11
+
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	const struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	const struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @mem2mem2_nodes: The array list of mtk_cam_video_device
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @dev_node_num: The number of supported V4L2 video device nodes
+ * @request_fd:	The file descriptor of request API
+ * @request_count: The buffer count of request API
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int dev_node_num;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..196aaef3d854
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1086 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENINF_PAD_SRC			4
+#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	cam_dev->sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count > 0) {
+			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!cam_dev->sensor) {
+		dev_err(dev, "Sensor is not connected\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
+	if (ret)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_dev_queue_req_buffers(cam_dev);
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	isp_composer_stream(isp_ctx, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
+		pad, remote->index);
+
+	if (pad < cam_dev->dev_node_num)
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
+		&buf->daddr, &buf->scp_addr);
+
+	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		v4l2_buf->request_fd,
+		v4l2_buf->vb2_buf.index);
+
+	if (v4l2_buf->request_fd < 0) {
+		mtk_cam_dev_queue_buffers(vb);
+		return;
+	}
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = cam_dev->smem_dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -ENOLINK;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (node->desc.smem_alloc)
+			vbq->bidirectional = 1;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_SENINF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_P1_HUB_PAD_SINK,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(
+		&cam_dev->pdev->dev, &cam_dev->notifier,
+		sizeof(struct v4l2_async_subdev),
+		mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..73b36916da08
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-10  1:58     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and six video devices during
initialization. Moreover, it also connects with sensor and
senif drivers with V4L2 async APIs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependeds on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/
---
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 ++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 5 files changed, 2156 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..5a581ab65945
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 000000000000..dda8a7b161ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,758 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static const struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = cal_main_stream_stride(dev, width, pix_fmt,
+					     one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = cal_packed_out_stride(dev, width, pix_fmt,
+					    one_pixel_mode);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+}
+
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd:%d, frame_seq:%d state:%d\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&vdev->slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&vdev->slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&vdev->slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf)
+			continue;
+
+		/* TBD: use buf_init callback function */
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
+				cam_dev->smem_dev, buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
+			node,
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			&buf->daddr,
+			&buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_req_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+
+	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
+
+	/* v4l2 sub-device registration */
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 000000000000..410460de44fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,250 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_DEV_NODES		11
+
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	const struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	const struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @mem2mem2_nodes: The array list of mtk_cam_video_device
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @dev_node_num: The number of supported V4L2 video device nodes
+ * @request_fd:	The file descriptor of request API
+ * @request_count: The buffer count of request API
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int dev_node_num;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..196aaef3d854
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1086 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENINF_PAD_SRC			4
+#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	cam_dev->sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count > 0) {
+			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!cam_dev->sensor) {
+		dev_err(dev, "Sensor is not connected\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
+	if (ret)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_dev_queue_req_buffers(cam_dev);
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	isp_composer_stream(isp_ctx, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
+		pad, remote->index);
+
+	if (pad < cam_dev->dev_node_num)
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
+		&buf->daddr, &buf->scp_addr);
+
+	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		v4l2_buf->request_fd,
+		v4l2_buf->vb2_buf.index);
+
+	if (v4l2_buf->request_fd < 0) {
+		mtk_cam_dev_queue_buffers(vb);
+		return;
+	}
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = cam_dev->smem_dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -ENOLINK;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (node->desc.smem_alloc)
+			vbq->bidirectional = 1;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_SENINF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_P1_HUB_PAD_SINK,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(
+		&cam_dev->pdev->dev, &cam_dev->notifier,
+		sizeof(struct v4l2_async_subdev),
+		mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..73b36916da08
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-10  1:58     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and six video devices during
initialization. Moreover, it also connects with sensor and
senif drivers with V4L2 async APIs.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependeds on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10852957/
---
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   19 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.c |  758 ++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-dev.h |  250 ++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1086 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |   43 +
 5 files changed, 2156 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..5a581ab65945
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,19 @@
+#
+# Copyright (C) 2018 MediaTek Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+
+mtk-cam-isp-objs += \
+	mtk_cam.o mtk_cam-dev.o \
+	mtk_cam-ctrl.o mtk_cam-scp.o \
+	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
new file mode 100644
index 000000000000..dda8a7b161ee
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
@@ -0,0 +1,758 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
+	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
+	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
+	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
+	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_videoc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
+	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
+static struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = MEDIA_LNK_FL_DYNAMIC,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+static const struct mtk_cam_dev_queues_setting queues_setting = {
+	.output_node_descs = output_queues,
+	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
+	.capture_node_descs = capture_queues,
+	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
+};
+
+static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 0;
+	}
+}
+
+static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 8);
+	case two_pixel_mode:
+		return ALIGN(size, 4);
+	case one_pixel_mode:
+		return ALIGN(size, 2);
+	default:
+		break;
+	}
+	return 0;
+}
+
+static unsigned int align_packetd_out_size(__u32 size,
+					   unsigned int pix_mode,
+					   __u32 fmt)
+{
+	switch (pix_mode) {
+	case default_pixel_mode:
+	case four_pixel_mode:
+		return ALIGN(size, 16);
+	case two_pixel_mode:
+		return ALIGN(size, 8);
+	case one_pixel_mode:
+		if (fmt == V4L2_PIX_FMT_MTISP_F10)
+			return ALIGN(size, 4);
+		else
+			return ALIGN(size, 8);
+	default:
+		return ALIGN(size, 16);
+	}
+	return 0;
+}
+
+static __u32 cal_main_stream_stride(struct device *dev,
+				    __u32 width,
+				    __u32 pix_fmt,
+				    __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+	/* expand stride, instead of shrink width */
+	stride = align_main_stream_size(stride, pix_mode);
+
+	dev_dbg(dev,
+		"main width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+	return stride;
+}
+
+static __u32 cal_packed_out_stride(struct device *dev,
+				   __u32 width,
+				   __u32 pix_fmt,
+				   __u32 pix_mode)
+{
+	__u32 stride;
+	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+	/* expand stride, instead of shrink width */
+	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
+
+	dev_dbg(dev,
+		"packed width:%d, pix_mode:%d, stride:%d\n",
+		width, pix_mode, stride);
+
+	return stride;
+}
+
+static __u32 cal_img_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = cal_main_stream_stride(dev, width, pix_fmt,
+					     one_pixel_mode);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = cal_packed_out_stride(dev, width, pix_fmt,
+					    one_pixel_mode);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* The helper to configure the device context */
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < setting->total_output_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->output_node_descs[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < setting->total_capture_nodes; i++)
+		cam_dev->mem2mem2_nodes[node_idx++].desc =
+			setting->capture_node_descs[i];
+
+	cam_dev->dev_node_num = node_idx;
+}
+
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *fram_param)
+{
+	struct mtk_cam_dev_buffer *buf, *b0;
+
+	if (!cam_dev->streaming)
+		return 0;
+
+	dev_dbg(&cam_dev->pdev->dev,
+		"job recvied request fd:%d, frame_seq:%d state:%d\n",
+		fram_param->request_fd,
+		fram_param->frame_seq_no,
+		fram_param->state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vbb.sequence = fram_param->frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
+	}
+
+	return 0;
+}
+
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+
+	return 0;
+}
+
+/* Calcuate mplane pix format */
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = cal_img_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue_desc,
+				  struct v4l2_format *dest)
+{
+	struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_fmt_set_img(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+/* Get a free buffer from a video node */
+static struct mtk_cam_dev_buffer *
+mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
+{
+	struct mtk_cam_dev_buffer *buf;
+	struct mtk_cam_video_device *vdev;
+
+	if (node > cam_dev->dev_node_num || node < 0) {
+		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
+		return NULL;
+	}
+	vdev = &cam_dev->mem2mem2_nodes[node];
+
+	spin_lock(&vdev->slock);
+	buf = list_first_entry_or_null(&vdev->pending_list,
+				       struct mtk_cam_dev_buffer,
+				       list);
+	if (!buf) {
+		spin_unlock(&vdev->slock);
+		return NULL;
+	}
+	list_del(&buf->list);
+	spin_unlock(&vdev->slock);
+
+	return buf;
+}
+
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int node;
+	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_dev_start_param s_param;
+	struct mtk_cam_dev_buffer *buf;
+
+	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
+
+	if (!cam_dev->streaming) {
+		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
+		return 0;
+	}
+
+	/* Check all enabled nodes to collect its buffer  */
+	for (node = 0; node < mtk_cam_dev_node_num; node++) {
+		if (!cam_dev->mem2mem2_nodes[node].enabled)
+			continue;
+		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
+		if (!buf)
+			continue;
+
+		/* TBD: use buf_init callback function */
+		buf->daddr =
+			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
+			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
+				cam_dev->smem_dev, buf->daddr);
+		} else {
+			buf->scp_addr = 0;
+		}
+
+		dev_dbg(dev,
+			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
+			node,
+			buf->vbb.request_fd,
+			buf->vbb.vb2_buf.index,
+			buf->vbb.vb2_buf.state,
+			&buf->daddr,
+			&buf->scp_addr);
+
+		s_param.buffers[node] = buf;
+		s_param.request_fd = buf->vbb.request_fd;
+	}
+
+	/* Trigger en-queued job to driver */
+	mtk_isp_req_enqueue(dev, &s_param);
+
+	return 0;
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+
+	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
+
+	/* v4l2 sub-device registration */
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
new file mode 100644
index 000000000000..410460de44fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
@@ -0,0 +1,250 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#ifndef __MTK_CAM_DEV_H__
+#define __MTK_CAM_DEV_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_DEV_NODES		11
+
+#define MTK_CAM_P1_META_IN_0		0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT	1
+#define MTK_CAM_P1_PACKED_BIN_OUT	2
+#define MTK_CAM_P1_META_OUT_0		3
+#define MTK_CAM_P1_META_OUT_1		4
+#define MTK_CAM_P1_META_OUT_2		5
+#define MTK_CAM_P1_META_OUT_3		6
+#define MTK_CAM_P1_TOTAL_CAPTURE	6
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+};
+
+/* Attributes setup by device owner */
+struct mtk_cam_dev_queues_setting {
+	const struct mtk_cam_dev_node_desc *output_node_descs;
+	unsigned int total_output_nodes;
+	const struct mtk_cam_dev_node_desc *capture_node_descs;
+	unsigned int total_capture_nodes;
+};
+
+struct mtk_cam_dev_start_param {
+	int request_fd;
+	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
+};
+
+struct mtk_cam_dev_finish_param {
+	int request_fd;
+	unsigned int frame_seq_no;
+	unsigned int state;
+	struct list_head *list_buf;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	struct v4l2_format *fmts;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @mem2mem2_nodes: The array list of mtk_cam_video_device
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @dev_node_num: The number of supported V4L2 video device nodes
+ * @request_fd:	The file descriptor of request API
+ * @request_count: The buffer count of request API
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int dev_node_num;
+	int request_fd;
+	unsigned int request_count;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler);
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
+			     const struct mtk_cam_dev_queues_setting *setting);
+int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
+			   struct mtk_cam_dev_finish_param *param);
+int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				 __u32 frame_seq_no);
+void mtk_cam_dev_fmt_set_img(struct device *dev,
+			     struct v4l2_pix_format_mplane *dest_fmt,
+			     struct v4l2_pix_format_mplane *src_fmt,
+			     unsigned int node_id);
+struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
+void mtk_cam_dev_load_default_fmt(struct device *dev,
+				  struct mtk_cam_dev_node_desc *queue,
+				  struct v4l2_format *dest_fmt);
+void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
+				    struct v4l2_pix_format_mplane *dest_fmt,
+				    unsigned int node_id);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+#endif /* __MTK_CAM_DEV_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..196aaef3d854
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1086 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-common.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-dev.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_SENINF_PAD_SRC			4
+#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
+
+static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	cam_dev->request_fd = -1;
+	cam_dev->request_count = 0;
+
+	return mtk_isp_open(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
+				struct v4l2_subdev_fh *fh)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	return mtk_isp_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	cam_dev->sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count > 0) {
+			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!cam_dev->sensor) {
+		dev_err(dev, "Sensor is not connected\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
+	if (ret)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_dev_queue_req_buffers(cam_dev);
+	isp_composer_stream(isp_ctx, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		goto fail_sensor_off;
+	}
+
+	isp_composer_stream(isp_ctx, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	return -EPERM;
+}
+
+static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
+				   int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
+					  struct v4l2_fh *fh,
+					  struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_link_setup(struct media_entity *entity,
+			      const struct media_pad *local,
+	const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
+		pad, remote->index);
+
+	if (pad < cam_dev->dev_node_num)
+		cam_dev->mem2mem2_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
+		&buf->daddr, &buf->scp_addr);
+
+	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+	struct vb2_v4l2_buffer *v4l2_buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		v4l2_buf->request_fd,
+		v4l2_buf->vb2_buf.index);
+
+	if (v4l2_buf->request_fd < 0) {
+		mtk_cam_dev_queue_buffers(vb);
+		return;
+	}
+
+	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
+		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
+		mtk_cam_dev->request_count =
+			vb->req_obj.req->num_incomplete_objects;
+		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
+			v4l2_buf->request_fd,
+			vb->req_obj.req->num_incomplete_objects);
+	}
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_cam_dev->request_count--;
+
+	if (!mtk_cam_dev->request_count) {
+		mtk_cam_dev->request_fd = -1;
+		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
+	}
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count > 0)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+	else
+		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
+
+	if (node->desc.smem_alloc) {
+		alloc_devs[0] = cam_dev->smem_dev;
+		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
+	} else {
+		alloc_devs[0] = &cam_dev->pdev->dev;
+		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
+			alloc_devs[0]);
+	}
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Validate initialized num_planes & size[0] */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	/* Initialize buffer queue & locks */
+	INIT_LIST_HEAD(&node->pending_list);
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+
+	return 0;
+}
+
+static bool
+mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
+			    struct mtk_cam_video_device *except)
+{
+	unsigned int i;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+
+		if (node == except)
+			continue;
+		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
+			return false;
+	}
+
+	return true;
+}
+
+static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
+				       struct mtk_cam_video_device *node,
+				       enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	unsigned int i;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	for (i = 0; i < node->vbq.num_buffers; ++i)
+		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(node->vbq.bufs[i], state);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
+			node->id);
+		ret = -ENOLINK;
+		goto fail_return_bufs;
+	}
+
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
+			node->id, __func__);
+		goto fail_return_bufs;
+	}
+
+	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret < 0) {
+		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
+			node->id);
+		goto fail_stop_pipeline;
+	}
+	return 0;
+
+fail_stop_pipeline:
+	media_pipeline_stop(&node->vdev.entity);
+fail_return_bufs:
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+
+	/* Was this the first node with streaming disabled? */
+	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
+		/* Yes, really stop streaming now */
+		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+			dev_err(&cam_dev->pdev->dev,
+				"failed to stop streaming\n");
+	}
+	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
+			   struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	if (in_fmt->type != node->vbq.type)
+		return -EINVAL;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc,
+					     in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
+				       &in_fmt->fmt.pix_mp,
+				       node->id);
+
+	return 0;
+}
+
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_videoc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input)
+{
+	if (input->index > 0)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = IMG_MAX_WIDTH;
+		sizes->stepwise.min_width = IMG_MIN_WIDTH;
+		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
+		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
+		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
+		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
+		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
+		sizes->stepwise.step_height = 1;
+		sizes->stepwise.step_width = 1;
+	}
+
+	return 0;
+}
+
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->index > 0 || f->type != node->vbq.type)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+
+	return 0;
+}
+
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	/* Each node is dedicated to only one meta format */
+	if (f->type != node->vbq.type)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+/* subdev internal operations */
+static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
+	.open = mtk_cam_subdev_open,
+	.close = mtk_cam_subdev_close,
+};
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_subdev_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream = mtk_cam_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+/*
+ * Config node's video properties
+ * according to the device context requirement
+ */
+static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
+				 unsigned int node,
+				 struct video_device *vdev,
+				 struct v4l2_format *f)
+{
+	struct mtk_cam_dev_node_desc *node_desc =
+		&cam_dev->mem2mem2_nodes[node].desc;
+
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
+	f->type = node_desc->buf_type;
+	vdev->ioctl_ops = node_desc->ioctl_ops;
+
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     node_desc,
+				     f);
+}
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = vb2_request_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	int ret;
+
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, media_dev);
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_register(struct device *dev,
+			  struct media_device *media_dev,
+			  struct v4l2_device *v4l2_dev,
+			  struct v4l2_ctrl_handler *ctrl_handler)
+{
+	int ret;
+
+	/* Set up v4l2 device */
+	v4l2_dev->ctrl_handler = ctrl_handler;
+	v4l2_dev->mdev = media_dev;
+	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
+	ret = v4l2_device_register(dev, v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	return 0;
+
+fail_v4l2_dev:
+	media_device_unregister(media_dev);
+	media_device_cleanup(media_dev);
+
+	return ret;
+}
+
+int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int num_nodes = cam_dev->dev_node_num;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		goto fail_media_dev;
+	}
+
+	ret = mtk_cam_v4l2_register(dev,
+				    &cam_dev->media_dev,
+				    &cam_dev->v4l2_dev,
+				    NULL);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
+
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+
+	/* Create video nodes and links */
+	for (i = 0; i < num_nodes; i++) {
+		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
+		struct video_device *vdev = &node->vdev;
+		struct vb2_queue *vbq = &node->vbq;
+		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
+		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
+
+		cam_dev->subdev_pads[i].flags = output ?
+			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+		/* Initialize miscellaneous variables */
+		mutex_init(&node->lock);
+		spin_lock_init(&node->slock);
+		INIT_LIST_HEAD(&node->pending_list);
+
+		/* Initialize formats to default values */
+		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
+
+		/* Initialize media entities */
+		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+		if (ret) {
+			dev_err(dev, "failed initialize media pad:%d\n", ret);
+			goto fail_vdev_media_entity;
+		}
+		node->enabled = false;
+		node->id = i;
+		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+		vdev->entity.ops = NULL;
+
+		/* Initialize vbq */
+		vbq->type = node->vdev_fmt.type;
+		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+			vbq->io_modes = VB2_MMAP;
+		else
+			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+		if (node->desc.smem_alloc)
+			vbq->bidirectional = 1;
+		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+			vdev->entity.function =
+				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+		vbq->ops = &mtk_cam_vb2_ops;
+		vbq->mem_ops = &vb2_dma_contig_memops;
+		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+		/* Put the process hub sub device in the vb2 private data */
+		vbq->drv_priv = cam_dev;
+		vbq->lock = &node->lock;
+		vbq->supports_requests = true;
+
+		ret = vb2_queue_init(vbq);
+		if (ret) {
+			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Initialize vdev */
+		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+			 MTK_CAM_DEV_P1_NAME, node->desc.name);
+		vdev->release = video_device_release_empty;
+		vdev->fops = &mtk_cam_v4l2_fops;
+		vdev->lock = &node->lock;
+		vdev->v4l2_dev = &cam_dev->v4l2_dev;
+		vdev->queue = &node->vbq;
+		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+		/* Enable private control for image video devices */
+		if (node->desc.image) {
+			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+			vdev->ctrl_handler = &node->ctrl_handler;
+		}
+		video_set_drvdata(vdev, cam_dev);
+		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret) {
+			dev_err(dev, "failed to register vde:%d\n", ret);
+			goto fail_vdev;
+		}
+
+		/* Create link between video node and the subdev pad */
+		if (output) {
+			ret = media_create_pad_link(&vdev->entity, 0,
+						    &cam_dev->subdev.entity,
+						    i, link_flags);
+		} else {
+			ret = media_create_pad_link(&cam_dev->subdev.entity,
+						    i, &vdev->entity, 0,
+						    link_flags);
+		}
+		if (ret)
+			goto fail_link;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+	for (; i >= 0; i--) {
+fail_link:
+		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
+fail_vdev:
+		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
+fail_vdev_media_entity:
+		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+fail_media_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		dev = &cam_dev->mem2mem2_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_SENINF_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_P1_HUB_PAD_SINK,
+				    0);
+	if (ret)
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_dev_fwnode_parse(struct device *dev,
+				    struct v4l2_fwnode_endpoint *vep,
+				    struct v4l2_async_subdev *asd)
+{
+	return 0;
+}
+
+int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(
+		&cam_dev->pdev->dev, &cam_dev->notifier,
+		sizeof(struct v4l2_async_subdev),
+		mtk_cam_dev_fwnode_parse);
+	if (ret < 0)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..73b36916da08
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+int mtk_cam_videoc_querycap(struct file *file, void *fh,
+			    struct v4l2_capability *cap);
+int mtk_cam_enum_framesizes(struct file *filp, void *priv,
+			    struct v4l2_frmsizeenum *sizes);
+int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
+			    struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+int mtk_cam_videoc_try_fmt(struct file *file,
+			   void *fh, struct v4l2_format *in_fmt);
+int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+			      struct v4l2_input *input);
+int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
+int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
+int mtk_cam_meta_enum_format(struct file *file, void *fh,
+			     struct v4l2_fmtdesc *f);
+int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
+			      struct v4l2_format *f);
+int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
+				   const struct v4l2_event_subscription *sub);
+
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,09/11] media: platform: Add Mediatek ISP P1 device driver
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:58   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 3 files changed, 1655 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..342f0e0e9837
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			SOF_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_HW_FRAME_NUM	0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX	0x14dc
+#define REG_META1_VB2_INDEX	0x151c
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..fc874ec8f7f0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1206 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
+	struct vb2_buffer *vb;
+	struct vb2_v4l2_buffer *vbb;
+
+	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
+		return;
+
+	mutex_lock(vb2_queue->lock);
+	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
+		vbb = to_vb2_v4l2_buffer(vb);
+		if (vbb->request_fd < 0 &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vbb->vb2_buf.timestamp = ktime_get_ns();
+			vbb->sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+		}
+	}
+	mutex_unlock(vb2_queue->lock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d drop, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_event_frame_sync(cam_dev,
+						     event_data.frame_seq_no);
+			dev_dbg(dev,
+				"event IRQ:0x%x DMA:0x%x is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+		}
+
+		if (event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, sof_cnt:%d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irqstatus & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x/0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR:%s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory\n");
+		return ret;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+		p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret < 0) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enable_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enable_dma_ports = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma_ports |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
+		__func__, enable_dma_ports);
+
+	return enable_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		ret = isp_setup_scp_rproc(p1_dev);
+		if (ret)
+			goto scp_err;
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+		/* Check enabled DMAs which is configured by media setup */
+		isp_ctx->enable_dma_ports =
+			get_enable_dma_ports(p1_dev->cam_dev);
+
+		if (!isp_ctx->enable_dma_ports) {
+			dev_dbg(dev, "No DMAs are enabled\n");
+			ret = -EINVAL;
+			goto scp_err;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			goto ctx_err;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+
+		isp_composer_meta_config(&p1_dev->isp_ctx,
+					 isp_ctx->enable_dma_ports);
+	}
+
+	return 0;
+composer_err:
+	isp_uninit_context(p1_dev);
+ctx_err:
+	pm_runtime_put_sync(dev);
+scp_err:
+	atomic_dec_return(&isp_ctx->isp_user_cnt);
+	return ret;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma_ports, idx;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma_ports = isp_ctx->enable_dma_ports;
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_port,
+		    struct mtk_cam_dev_buffer *buffer)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
+
+	return 0;
+}
+
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6cf8bb4ba93a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,300 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-scp.h"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int enable_dma_ports;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_config - output image & meta data configuration.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_config(struct device *dev);
+
+/**
+ * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase);
+
+/**
+ * mtk_isp_enqueue - enqueue a single frame to ISP driver
+ * for non-per-frame DMA.
+ *
+ * @dev:	isp device.
+ * @buffer: pointer to &struct mtk_cam_dev_buffer.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_idx,
+		    struct mtk_cam_dev_buffer *buffer);
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 3 files changed, 1655 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..342f0e0e9837
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			SOF_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_HW_FRAME_NUM	0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX	0x14dc
+#define REG_META1_VB2_INDEX	0x151c
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..fc874ec8f7f0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1206 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
+	struct vb2_buffer *vb;
+	struct vb2_v4l2_buffer *vbb;
+
+	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
+		return;
+
+	mutex_lock(vb2_queue->lock);
+	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
+		vbb = to_vb2_v4l2_buffer(vb);
+		if (vbb->request_fd < 0 &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vbb->vb2_buf.timestamp = ktime_get_ns();
+			vbb->sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+		}
+	}
+	mutex_unlock(vb2_queue->lock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d drop, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_event_frame_sync(cam_dev,
+						     event_data.frame_seq_no);
+			dev_dbg(dev,
+				"event IRQ:0x%x DMA:0x%x is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+		}
+
+		if (event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, sof_cnt:%d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irqstatus & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x/0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR:%s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory\n");
+		return ret;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+		p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret < 0) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enable_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enable_dma_ports = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma_ports |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
+		__func__, enable_dma_ports);
+
+	return enable_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		ret = isp_setup_scp_rproc(p1_dev);
+		if (ret)
+			goto scp_err;
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+		/* Check enabled DMAs which is configured by media setup */
+		isp_ctx->enable_dma_ports =
+			get_enable_dma_ports(p1_dev->cam_dev);
+
+		if (!isp_ctx->enable_dma_ports) {
+			dev_dbg(dev, "No DMAs are enabled\n");
+			ret = -EINVAL;
+			goto scp_err;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			goto ctx_err;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+
+		isp_composer_meta_config(&p1_dev->isp_ctx,
+					 isp_ctx->enable_dma_ports);
+	}
+
+	return 0;
+composer_err:
+	isp_uninit_context(p1_dev);
+ctx_err:
+	pm_runtime_put_sync(dev);
+scp_err:
+	atomic_dec_return(&isp_ctx->isp_user_cnt);
+	return ret;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma_ports, idx;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma_ports = isp_ctx->enable_dma_ports;
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_port,
+		    struct mtk_cam_dev_buffer *buffer)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
+
+	return 0;
+}
+
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6cf8bb4ba93a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,300 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-scp.h"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int enable_dma_ports;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_config - output image & meta data configuration.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_config(struct device *dev);
+
+/**
+ * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase);
+
+/**
+ * mtk_isp_enqueue - enqueue a single frame to ISP driver
+ * for non-per-frame DMA.
+ *
+ * @dev:	isp device.
+ * @buffer: pointer to &struct mtk_cam_dev_buffer.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_idx,
+		    struct mtk_cam_dev_buffer *buffer);
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
 3 files changed, 1655 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..342f0e0e9837
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT	BIT(0)
+#define CMOS_EN_BIT	BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST	BIT(0)
+#define HW_PASS1_DON_ST	BIT(11)
+#define SOF_INT_ST	BIT(12)
+#define SW_PASS1_DON_ST	BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST	BIT(4)
+#define TG_GBERR_ST	BIT(5)
+#define CQ_CODE_ERR_ST	BIT(6)
+#define CQ_APB_ERR_ST	BIT(7)
+#define CQ_VS_ERR_ST	BIT(8)
+#define AMX_ERR_ST	BIT(15)
+#define RMX_ERR_ST	BIT(16)
+#define BMX_ERR_ST	BIT(17)
+#define RRZO_ERR_ST	BIT(18)
+#define AFO_ERR_ST	BIT(19)
+#define IMGO_ERR_ST	BIT(20)
+#define AAO_ERR_ST	BIT(21)
+#define PSO_ERR_ST	BIT(22)
+#define LCSO_ERR_ST	BIT(23)
+#define BNR_ERR_ST	BIT(24)
+#define LSCI_ERR_ST	BIT(25)
+#define DMA_ERR_ST	BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST	BIT(4)
+#define AFO_DONE_ST	BIT(5)
+#define AAO_DONE_ST	BIT(7)
+#define PSO_DONE_ST	BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM	( \
+			VS_INT_ST |\
+			SOF_INT_ST |\
+			HW_PASS1_DON_ST |\
+			SW_PASS1_DON_ST)
+
+/* IRQ Warning Mask */
+#define INT_ST_MASK_CAM_WARN	(\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				PSO_ERR_ST | \
+				LCSO_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR	(\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				CQ_CODE_ERR_ST |\
+				CQ_APB_ERR_ST |\
+				CQ_VS_ERR_ST |\
+				BNR_ERR_ST |\
+				RMX_ERR_ST |\
+				BMX_ERR_ST |\
+				BNR_ERR_ST |\
+				LSCI_ERR_ST |\
+				DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM	(\
+				SOF_INT_ST |\
+				SW_PASS1_DON_ST |\
+				VS_INT_ST |\
+				TG_ERR_ST |\
+				TG_GBERR_ST |\
+				RRZO_ERR_ST |\
+				AFO_ERR_ST |\
+				IMGO_ERR_ST |\
+				AAO_ERR_ST |\
+				DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM	(\
+			AFO_DONE_ST |\
+			AAO_DONE_ST |\
+			PSO_DONE_ST |\
+			FLKO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN		0x0004
+#define REG_CTL_DMA_EN		0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2		0x0018
+#define REG_CTL_RAW_INT_EN	0x0020
+#define REG_CTL_RAW_INT_STAT	0x0024
+#define REG_CTL_RAW_INT2_STAT	0x0034
+#define REG_CTL_RAW_INT3_STAT	0x00c4
+#define REG_CTL_TWIN_STAT	0x0050
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_SEN_GRAB_PIX	0x0238
+#define REG_TG_SEN_GRAB_LIN	0x023c
+#define REG_TG_VF_CON		0x0234
+#define REG_TG_SUB_PERIOD	0x02a4
+
+#define REG_IMGO_BASE_ADDR	0x1020
+#define REG_RRZO_BASE_ADDR	0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT	0x1360
+#define REG_RRZO_ERR_STAT	0x1364
+#define REG_AAO_ERR_STAT	0x1368
+#define REG_AFO_ERR_STAT	0x136c
+#define REG_LCSO_ERR_STAT	0x1370
+#define REG_UFEO_ERR_STAT	0x1374
+#define REG_PDO_ERR_STAT	0x1378
+#define REG_BPCI_ERR_STAT	0x137c
+#define REG_LSCI_ERR_STAT	0x1384
+#define REG_PDI_ERR_STAT	0x138c
+#define REG_LMVO_ERR_STAT	0x1390
+#define REG_FLKO_ERR_STAT	0x1394
+#define REG_PSO_ERR_STAT	0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR	0x0198
+#define REG_HW_FRAME_NUM	0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX	0x14dc
+#define REG_META1_VB2_INDEX	0x151c
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..fc874ec8f7f0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1206 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* list of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
+			   unsigned int request_fd,
+			   unsigned int frame_seq_no,
+			   struct list_head *list_buf,
+			   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev_finish_param fram_param;
+
+	fram_param.list_buf = list_buf;
+	fram_param.request_fd = request_fd;
+	fram_param.frame_seq_no = frame_seq_no;
+	fram_param.state = state;
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
+		fram_param.request_fd,
+		fram_param.frame_seq_no);
+	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
+	struct vb2_buffer *vb;
+	struct vb2_v4l2_buffer *vbb;
+
+	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
+		return;
+
+	mutex_lock(vb2_queue->lock);
+	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
+		vbb = to_vb2_v4l2_buffer(vb);
+		if (vbb->request_fd < 0 &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vbb->vb2_buf.timestamp = ktime_get_ns();
+			vbb->sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+		}
+	}
+	mutex_unlock(vb2_queue->lock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_isp_notify(isp_ctx,
+				       framejob->request_fd,
+				       framejob->frame_seq_no,
+				       &framejob->list_buf,
+				       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d drop, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & VS_INT_ST) {
+			/* Notify specific HW events to user space */
+			mtk_cam_dev_event_frame_sync(cam_dev,
+						     event_data.frame_seq_no);
+			dev_dbg(dev,
+				"event IRQ:0x%x DMA:0x%x is sent\n",
+				event_data.irq_status_mask,
+				event_data.dma_status_mask);
+		}
+
+		if (event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int cq_addr_index;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	if (cq_num > frame_num) {
+		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
+
+		writel(base_addr +
+			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
+			isp_dev->regs + REG_CQ_THR0_BASEADDR);
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
+			cq_num, frame_num, cq_addr_index);
+	} else {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+	}
+
+	isp_dev->sof_count += 1;
+
+	return cq_num;
+}
+
+static int irq_handle_notify_event(struct isp_device *isp_dev,
+				   unsigned int irqstatus,
+				   unsigned int dmastatus)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask |=
+		(irqstatus & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask |=
+		(dmastatus & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(isp_dev->dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irqstatus & INT_ST_MASK_CAM),
+		(dmastatus & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+
+	return 0;
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cardinalnum, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev->streaming)
+		return IRQ_HANDLED;
+
+	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (irqstatus == 0)
+		return IRQ_HANDLED;
+
+	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
+	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
+	irqstatus = irqstatus & INT_ST_MASK_CAM;
+
+	/* sof , done order check . */
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
+		dev_warn(dev,
+			 "isp sof_don block, sof_cnt:%d\n",
+			 isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue ready frame */
+		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irqstatus & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+
+		if ((irqstatus & INT_ST_MASK_CAM) ||
+		    (dmastatus & DMA_ST_MASK_CAM))
+			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
+	}
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	if (irqstatus & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	if (irqstatus & SW_PASS1_DON_ST) {
+		int num = atomic_dec_return(&isp_ctx->composing_frame);
+
+		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible
+				(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	/* check ISP error status */
+	if (errstatus) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x/0x%x\n",
+			irqstatus, warnstatus, errstatus);
+
+		/* show DMA errors in detail */
+		if (errstatus & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irqstatus & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cardinalnum,
+			isp_dev->sof_count,
+			irqstatus,
+			dmastatus,
+			hw_frame_num,
+			cq_num);
+	return IRQ_HANDLED;
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
+				      p1_dev->isp_clk.clk_list);
+	if (ret < 0)
+		goto clk_err;
+	return 0;
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+
+	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
+	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
+				   p1_dev->isp_clk.clk_list);
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int ret;
+	unsigned int i;
+
+	/* Allocate context */
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	p1_dev->isp_devs =
+		devm_kzalloc(dev,
+			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
+			     GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	p1_dev->cam_dev =
+		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
+	if (!p1_dev->isp_devs)
+		return -ENOMEM;
+
+	/* iomap registers */
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_info(dev, "cam%u, map_addr=0x%lx\n",
+			 i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* reg & interrupts index is shifted with 1  */
+			isp_dev->irq = platform_get_irq(pdev, i - 1);
+			if (isp_dev->irq > 0) {
+				ret = devm_request_irq(dev, isp_dev->irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						isp_dev->irq);
+					return ret;
+				}
+				dev_info(dev, "Registered irq=%d, ISR:%s\n",
+					 isp_dev->irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_clk.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_clk.num_clks,
+			     sizeof(*p1_dev->isp_clk.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_clk.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
+		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_clk.num_clks,
+				p1_dev->isp_clk.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory\n");
+		return ret;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
+
+	return 0;
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int usercount, module;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* disable VF */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, The TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct isp_device *isp_dev;
+	unsigned int reg_val;
+	int module, usercount;
+
+	module = p1_dev->isp_ctx.isp_hw_module;
+	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
+
+	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
+
+	/* If no user count, no further action */
+	if (!usercount)
+		return 0;
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev->streaming) {
+		isp_dev = &p1_dev->isp_devs[module];
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+	return 0;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
+		p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret < 0) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+	return 0;
+}
+
+static int isp_uninit_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	atomic_set(&isp_ctx->isp_user_cnt, 0);
+
+	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	return 0;
+}
+
+static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enable_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enable_dma_ports = 0;
+	for (i = 0; i < cam_dev->dev_node_num; i++) {
+		if (cam_dev->mem2mem2_nodes[i].enabled)
+			enable_dma_ports |=
+				cam_dev->mem2mem2_nodes[i].desc.dma_port;
+	}
+	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
+		__func__, enable_dma_ports);
+
+	return enable_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return raw_pxl_id_b;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return raw_pxl_id_gb;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return raw_pxl_id_gr;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return raw_pxl_id_r;
+	default:
+		return raw_pxl_id_b;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return img_fmt_bayer8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return img_fmt_bayer10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return img_fmt_bayer12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return img_fmt_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return img_fmt_bayer8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return img_fmt_fg_bayer8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return img_fmt_bayer10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return img_fmt_fg_bayer10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return img_fmt_bayer12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return img_fmt_fg_bayer12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return img_fmt_bayer14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return img_fmt_fg_bayer14;
+	default:
+		return img_fmt_unknown;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	case V4L2_PIX_FMT_MTISP_U8:
+	case V4L2_PIX_FMT_MTISP_U10:
+	case V4L2_PIX_FMT_MTISP_U12:
+	case V4L2_PIX_FMT_MTISP_U14:
+		return 16;
+	default:
+		return 10;
+	}
+}
+
+static void composer_deinit_done_cb(void *data)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+
+	disable_sys_clock(p1_dev);
+	/* Notify PM */
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_open(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
+	int ret;
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
+
+	if (usercount == 1) {
+		ret = isp_setup_scp_rproc(p1_dev);
+		if (ret)
+			goto scp_err;
+
+		/* ISP HW INIT */
+		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+		/* Use pure RAW as default HW path */
+		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+		/* Check enabled DMAs which is configured by media setup */
+		isp_ctx->enable_dma_ports =
+			get_enable_dma_ports(p1_dev->cam_dev);
+
+		if (!isp_ctx->enable_dma_ports) {
+			dev_dbg(dev, "No DMAs are enabled\n");
+			ret = -EINVAL;
+			goto scp_err;
+		}
+
+		pm_runtime_get_sync(dev);
+
+		ret = isp_init_context(p1_dev);
+		if (ret)
+			goto ctx_err;
+		ret = isp_composer_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+		ret = isp_composer_hw_init(isp_ctx);
+		if (ret)
+			goto composer_err;
+
+		isp_composer_meta_config(&p1_dev->isp_ctx,
+					 isp_ctx->enable_dma_ports);
+	}
+
+	return 0;
+composer_err:
+	isp_uninit_context(p1_dev);
+ctx_err:
+	pm_runtime_put_sync(dev);
+scp_err:
+	atomic_dec_return(&isp_ctx->isp_user_cnt);
+	return ret;
+}
+
+int mtk_isp_release(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
+		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
+		isp_uninit_context(p1_dev);
+	}
+
+	dev_dbg(dev, "%s usercount=%d\n", __func__,
+		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_format;
+	unsigned int sd_width, sd_height;
+	unsigned int enable_dma_ports, idx;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enable_dma_ports = isp_ctx->enable_dma_ports;
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
+
+	/* sensor config */
+	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor,
+			       pad, get_fmt, NULL, &sd_format);
+
+	if (ret) {
+		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_format.format.width, sd_format.format.height,
+		sd_format.format.code, sd_format.format.field,
+		sd_format.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
+	/* support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_format.format.code);
+	config_param.cfg_in_param.img_fmt =
+		get_sensor_fmt(sd_format.format.code);
+	config_param.cfg_in_param.crop.width = sd_format.format.width;
+	config_param.cfg_in_param.crop.height = sd_format.format.height;
+	sd_width = sd_format.format.width;
+	sd_height = sd_format.format.height;
+
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
+		struct v4l2_format *imgo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_param.cfg_main_param.bypass = 0;
+
+		config_param.cfg_main_param.output.img_fmt =
+			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.pixel_byte =
+			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_main_param.output.size.w =
+			imgo_fmt->fmt.pix_mp.width;
+		config_param.cfg_main_param.output.size.h =
+			imgo_fmt->fmt.pix_mp.height;
+
+		config_param.cfg_main_param.output.size.stride =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_main_param.output.size.xsize =
+			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_main_param.output.crop.left = 0x0;
+		config_param.cfg_main_param.output.crop.top = 0x0;
+
+		config_param.cfg_main_param.output.crop.width = sd_width;
+		config_param.cfg_main_param.output.crop.height = sd_height;
+
+		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
+			  imgo_fmt->fmt.pix_mp.height > sd_height,
+			  "img out:%d:%d in:%d:%d",
+			  imgo_fmt->fmt.pix_mp.width,
+			  imgo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev,
+			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
+			config_param.cfg_main_param.output.pixel_byte,
+			config_param.cfg_main_param.output.img_fmt,
+			config_param.cfg_main_param.pure_raw);
+		dev_dbg(dev,
+			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_main_param.output.size.w,
+			config_param.cfg_main_param.output.size.h,
+			config_param.cfg_main_param.output.size.stride,
+			config_param.cfg_main_param.output.size.xsize,
+			config_param.cfg_main_param.output.crop.width,
+			config_param.cfg_main_param.output.crop.height);
+	} else {
+		config_param.cfg_main_param.bypass = 1;
+	}
+
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
+		struct v4l2_format *rrzo_fmt =
+			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
+
+		config_param.cfg_resize_param.bypass = 0;
+		config_param.cfg_resize_param.output.img_fmt =
+			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.pixel_byte =
+			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
+		config_param.cfg_resize_param.output.size.w =
+			rrzo_fmt->fmt.pix_mp.width;
+		config_param.cfg_resize_param.output.size.h =
+			rrzo_fmt->fmt.pix_mp.height;
+		config_param.cfg_resize_param.output.size.stride =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+		config_param.cfg_resize_param.output.size.xsize =
+			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+		config_param.cfg_resize_param.output.crop.left = 0x0;
+		config_param.cfg_resize_param.output.crop.top = 0x0;
+		config_param.cfg_resize_param.output.crop.width = sd_width;
+		config_param.cfg_resize_param.output.crop.height = sd_height;
+
+		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
+			  rrzo_fmt->fmt.pix_mp.height > sd_height,
+			  "rrz out:%d:%d in:%d:%d",
+			  rrzo_fmt->fmt.pix_mp.width,
+			  rrzo_fmt->fmt.pix_mp.height,
+			  sd_width,
+			  sd_height);
+
+		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
+			config_param.cfg_resize_param.output.pixel_byte,
+			config_param.cfg_resize_param.output.img_fmt);
+		dev_dbg(dev,
+			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
+			config_param.cfg_resize_param.output.size.w,
+			config_param.cfg_resize_param.output.size.h,
+			config_param.cfg_resize_param.output.size.stride,
+			config_param.cfg_resize_param.output.size.xsize,
+			config_param.cfg_resize_param.output.crop.width,
+			config_param.cfg_resize_param.output.crop.height);
+	} else {
+		config_param.cfg_resize_param.bypass = 1;
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
+
+	isp_composer_hw_config(isp_ctx, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+	return 0;
+}
+
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_port,
+		    struct mtk_cam_dev_buffer *buffer)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
+
+	return 0;
+}
+
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct mtk_cam_dev_buffer **bundle_buffers;
+	unsigned int i, idx;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	bundle_buffers = &frameparamsbase->buffers[0];
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->request_fd = frameparamsbase->request_fd;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	idx = MTK_CAM_P1_META_IN_0;
+	if (bundle_buffers[idx]) {
+		frameparams.tuning_addr.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.tuning_addr.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Image output */
+	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[0].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[0].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "main stream address iova:0x%x\n",
+			frameparams.img_dma_buffers[0].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Resize output */
+	idx = MTK_CAM_P1_PACKED_BIN_OUT;
+	if (bundle_buffers[idx]) {
+		frameparams.img_dma_buffers[1].buffer.iova =
+			bundle_buffers[idx]->daddr;
+		frameparams.img_dma_buffers[1].buffer.scp_addr =
+			bundle_buffers[idx]->scp_addr;
+		dev_dbg(dev, "packed out address iova:0x%x\n",
+			frameparams.img_dma_buffers[1].buffer.iova);
+		list_add_tail(&bundle_buffers[idx]->list,
+			      &framejob->list_buf);
+	}
+
+	/* Meta output DMAs */
+	for (i = 0; i < MAX_META_DMA_NODES; i++) {
+		idx = MTK_CAM_P1_META_OUT_0 + i;
+		if (bundle_buffers[idx]) {
+			frameparams.meta_addrs[i].iova =
+			  bundle_buffers[idx]->daddr;
+			frameparams.meta_addrs[i].scp_addr =
+			  bundle_buffers[idx]->scp_addr;
+			list_add_tail(&bundle_buffers[idx]->list,
+				      &framejob->list_buf);
+		} else {
+			frameparams.meta_addrs[i].iova = 0;
+			frameparams.meta_addrs[i].scp_addr = 0;
+		}
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		frameparamsbase->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6cf8bb4ba93a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,300 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Ryan Yu <ryan.yu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-dev.h"
+#include "mtk_cam-scp.h"
+
+#define CAM_A_MAX_WIDTH		3328U
+#define CAM_A_MAX_HEIGHT	2496U
+#define CAM_B_MAX_WIDTH		5376U
+#define CAM_B_MAX_HEIGHT	4032U
+
+#define CAM_MIN_WIDTH		80U
+#define CAM_MIN_HEIGHT		60U
+
+#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_PDO		BIT(6)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_RSSO		BIT(9)
+#define R_PSO		BIT(10)
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_DATA_BUF_SIZE		4
+#define COMPOSRE_EVENT_BUF_SIZE		4
+
+#define CQ_ADDRESS_OFFSET		0x640
+#define CQ_BUFFER_COUNT			3
+
+#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+enum {
+	img_fmt_unknown		= 0x0000,
+	img_fmt_raw_start	= 0x2200,
+	img_fmt_bayer8		= img_fmt_raw_start,
+	img_fmt_bayer10,
+	img_fmt_bayer12,
+	img_fmt_bayer14,
+	img_fmt_fg_bayer8,
+	img_fmt_fg_bayer10,
+	img_fmt_fg_bayer12,
+	img_fmt_fg_bayer14,
+};
+
+enum {
+	raw_pxl_id_b   = 0,
+	raw_pxl_id_gb,
+	raw_pxl_id_gr,
+	raw_pxl_id_r
+};
+
+enum {
+	default_pixel_mode = 0,
+	one_pixel_mode,
+	two_pixel_mode,
+	four_pixel_mode,
+	pixel_mode_num,
+};
+
+enum mtk_isp_scp_ipi_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_ipi_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+struct isp_clk_struct {
+	int num_clks;
+	struct clk_bulk_data *clk_list;
+};
+
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	int irq;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	struct mutex composer_tx_lock; /* isp composer work protection */
+
+	struct isp_thread composer_rx_thread;
+	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
+	atomic_t composer_evts_start;
+	atomic_t composer_evts_end;
+	spinlock_t composer_evts_lock; /* SCP events protection */
+	/* increase after ipi */
+	atomic_t ipi_occupied;
+	/* increase after frame enqueue */
+	atomic_t composing_frame;
+	/* current composed frame id */
+	atomic_t composed_frame_id;
+
+	struct isp_queue p1_enqueue_list;
+
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	struct sg_table sgtable;
+
+	/* increase after open, decrease when close */
+	atomic_t isp_user_cnt;
+	/* frame sequence number, increase per en-queue*/
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int enable_dma_ports;
+
+	void (*composer_deinit_donecb)(void *isp_ctx);
+
+	struct list_head list;
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+
+	/* for SCP driver  */
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct isp_clk_struct isp_clk;
+	struct mtk_cam_dev *cam_dev;
+	struct isp_device *isp_devs;
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma);
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param);
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data));
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type);
+
+/**
+ * mtk_isp_open - open isp driver and initialize related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_open(struct device *dev);
+
+/**
+ * mtk_isp_release - release isp driver and related resources.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_release(struct device *dev);
+
+/**
+ * mtk_isp_config - output image & meta data configuration.
+ *
+ * @dev:	isp device.
+ *
+ */
+int mtk_isp_config(struct device *dev);
+
+/**
+ * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
+ *
+ * @dev:	isp device.
+ * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
+ *
+ */
+int mtk_isp_req_enqueue(struct device *dev,
+			struct mtk_cam_dev_start_param *frameparamsbase);
+
+/**
+ * mtk_isp_enqueue - enqueue a single frame to ISP driver
+ * for non-per-frame DMA.
+ *
+ * @dev:	isp device.
+ * @buffer: pointer to &struct mtk_cam_dev_buffer.
+ *
+ */
+int mtk_isp_enqueue(struct device *dev,
+		    unsigned int dma_idx,
+		    struct mtk_cam_dev_buffer *buffer);
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 10/11] media: platform: Add Mediatek ISP P1 SCP communication
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:58   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 481 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++
 2 files changed, 688 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..cd10a1c43e0b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,481 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "scp mem addr:%pad size:%u MiB\n", &isp_ctx->scp_mem_pa,
+		(size / SZ_1M));
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = PAGE_ALIGN(size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+fail_pages_alloc:
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg, &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied < ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	memcpy(&composer_tx_cmd.cfg_meta_out_param, &dma, sizeof(dma));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.frameparam,
+	       config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.is_stream_on, &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..54025dfdb4e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_NODES	4
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,10/11] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 481 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++
 2 files changed, 688 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..cd10a1c43e0b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,481 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "scp mem addr:%pad size:%u MiB\n", &isp_ctx->scp_mem_pa,
+		(size / SZ_1M));
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = PAGE_ALIGN(size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+fail_pages_alloc:
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg, &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied < ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	memcpy(&composer_tx_cmd.cfg_meta_out_param, &dma, sizeof(dma));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.frameparam,
+	       config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.is_stream_on, &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..54025dfdb4e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_NODES	4
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 10/11] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch dependents on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10872547/
---
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 481 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++
 2 files changed, 688 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..cd10a1c43e0b
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,481 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static int isp_composer_dma_sg_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size, size_align;
+	struct sg_table *sgt;
+	struct page **pages;
+	unsigned int n_pages, i;
+	int ret;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = scp_get_reserve_mem_size(SCP_ISP_MEM_ID);
+
+	dev_dbg(dev, "scp mem addr:%pad size:%u MiB\n", &isp_ctx->scp_mem_pa,
+		(size / SZ_1M));
+
+	/* get iova address */
+	sgt = &isp_ctx->sgtable;
+	sg_alloc_table(sgt, 1, GFP_KERNEL);
+
+	size_align = PAGE_ALIGN(size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		goto fail_pages_alloc;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(isp_ctx->scp_mem_pa + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages,
+					0, size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to allocate sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	isp_ctx->scp_mem_iova = sg_dma_address(sgt->sgl);
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+fail_pages_alloc:
+	return -ENOMEM;
+}
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+		atomic_dec(&isp_ctx->composer_txlist.queue_cnt);
+	}
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	mutex_destroy(&isp_ctx->composer_tx_lock);
+
+	dma_unmap_sg_attrs(&p1_dev->pdev->dev, isp_ctx->sgtable.sgl,
+			   isp_ctx->sgtable.orig_nents,
+			   DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(&isp_ctx->sgtable);
+
+	if (!IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	isp_ctx->composer_deinit_donecb(isp_ctx);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			mutex_lock(&isp_ctx->composer_tx_lock);
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			mutex_unlock(&isp_ctx->composer_tx_lock);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+			kfree(isp_composer_work);
+			continue;
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_rx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd ipi_msg;
+	int ret;
+	unsigned int ipi_queue_occupied, i;
+	unsigned long flags;
+	atomic_t *evts_end = &isp_ctx->composer_evts_end;
+	atomic_t *evts_start = &isp_ctx->composer_evts_start;
+	u8 ack_cmd_id;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->composer_rx_thread.wq,
+					       (atomic_read(evts_end) !=
+					       atomic_read(evts_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		if (ret == ERESTARTSYS) {
+			dev_err(dev, "interrupted by a signal!\n");
+			continue;
+		}
+
+		spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+		i = atomic_read(evts_start);
+		memcpy(&ipi_msg, &isp_ctx->composer_evts[i],
+		       sizeof(struct mtk_isp_scp_p1_cmd));
+		atomic_set(evts_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+		switch (ipi_msg.cmd_id) {
+		case ISP_CMD_ACK:
+			ipi_queue_occupied =
+				atomic_dec_return(&isp_ctx->ipi_occupied);
+			if (ipi_queue_occupied < ISP_COMPOSING_MAX_NUM)
+				wake_up_interruptible
+					(&isp_ctx->composer_tx_thread.wq);
+
+			ack_cmd_id = ipi_msg.ack_info.cmd_id;
+			if (ack_cmd_id == ISP_CMD_FRAME_ACK) {
+				dev_dbg(dev,
+					"%s frame %d ack\n",
+					__func__,
+					ipi_msg.ack_info.frame_seq_no);
+				atomic_set(&isp_ctx->composed_frame_id,
+					   ipi_msg.ack_info.frame_seq_no);
+			} else {
+				dev_dbg(dev, "%s cmd id: %d",
+					__func__,
+					ack_cmd_id);
+				if (ack_cmd_id == ISP_CMD_DEINIT) {
+					isp_composer_deinit(isp_ctx);
+					isp_ctx->composer_rx_thread.thread =
+						NULL;
+					return -1;
+				}
+			}
+			break;
+		default:
+			break;
+		};
+	}
+	return ret;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct mtk_isp_scp_p1_cmd *ipi_msg_ptr;
+	unsigned long flags;
+	unsigned int i;
+
+	ipi_msg_ptr = (struct mtk_isp_scp_p1_cmd *)data;
+	isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+
+	spin_lock_irqsave(&isp_ctx->composer_evts_lock, flags);
+	i = atomic_read(&isp_ctx->composer_evts_end);
+	memcpy(&isp_ctx->composer_evts[i], data,
+	       sizeof(struct mtk_isp_scp_p1_cmd));
+	atomic_set(&isp_ctx->composer_evts_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->composer_evts_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->composer_rx_thread.wq);
+}
+
+int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	if (!isp_ctx->composer_tx_thread.thread) {
+		mutex_init(&isp_ctx->composer_tx_lock);
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, (void *)isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+
+	if (!isp_ctx->composer_rx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_rx_thread.wq);
+		isp_ctx->composer_rx_thread.thread =
+			kthread_run(isp_composer_rx_work, (void *)isp_ctx,
+				    "isp_composer_rx");
+		if (IS_ERR(isp_ctx->composer_rx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_rx_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+
+	atomic_set(&isp_ctx->composer_evts_start, 0);
+	atomic_set(&isp_ctx->composer_evts_end, 0);
+	spin_lock_init(&isp_ctx->composer_evts_lock);
+
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->cmd_queued, 0);
+
+	return 0;
+}
+
+void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
+			  void *data,
+			  enum mtk_isp_scp_ipi_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct img_buffer frameparam;
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	int ret;
+
+	ret = isp_composer_dma_sg_init(isp_ctx);
+	if (ret)
+		return ret;
+
+	frameparam.scp_addr = isp_ctx->scp_mem_pa;
+	frameparam.iova = isp_ctx->scp_mem_iova;
+
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	memcpy(&composer_tx_cmd.frameparam.cq_addr, &frameparam,
+	       sizeof(struct img_buffer));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+
+	return 0;
+}
+
+void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	memcpy(&composer_tx_cmd.cfg_meta_out_param, &dma, sizeof(dma));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.frameparam,
+	       config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	memcpy(&composer_tx_cmd.is_stream_on, &on, sizeof(on));
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
+			    void (*donecb)(void *data))
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_ctx->composer_deinit_donecb = donecb;
+	isp_composer_enqueue(isp_ctx, &composer_tx_cmd, SCP_ISP_CMD);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..54025dfdb4e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Seraph Huang <seraph.huang@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/remoteproc.h>
+#include <linux/types.h>
+
+#define MAX_IMG_DMA_PORT	2
+#define MAX_META_DMA_NODES	4
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer tuning_addr;
+	struct p1_img_output img_dma_buffers[MAX_IMG_DMA_PORT];
+	struct img_buffer meta_addrs[MAX_META_DMA_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_CMD {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 11/11] media: platform: Add Mediatek ISP P1 shared memory device
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-05-10  1:58   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     | 297 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  28 ++
 2 files changed, 325 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
new file mode 100644
index 000000000000..b7c8d3a00c2c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
@@ -0,0 +1,297 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/cacheflush.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-smem.h"
+
+static struct reserved_mem *cam_reserved_smem;
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%d pages:%u\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+
+	return 0;
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+
+	if (ret)
+		dev_err(dev, "Unable to declare smem  memory:\n", ret);
+	else
+		dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+			 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+	of_reserved_mem_device_release(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int rc = 0;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	rc = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev->smem_dev = smem_dev->dev;
+
+	if (cam_reserved_smem) {
+		smem_dev->smem_base = cam_reserved_smem->base;
+		smem_dev->smem_size = cam_reserved_smem->size;
+		dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+			 __func__,
+			 smem_dev->dev,
+			 &smem_dev->smem_base,
+			 (smem_dev->smem_size / SZ_1M));
+		mtk_cam_reserved_drm_sg_init(smem_dev);
+		mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+	}
+
+	return 0;
+}
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: no-map are not supported\n");
+		return -EINVAL;
+	}
+
+	cam_reserved_smem = rmem;
+
+	pr_info("%s:created DMA memory pool at %pa, size %u MiB\n",
+		__func__, &rmem->base, (rmem->size / SZ_1M));
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam-smem",
+		       mtk_cam_smem_dma_setup);
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..44fb876c87d5
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,V2,11/11] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Jungo Lin

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     | 297 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  28 ++
 2 files changed, 325 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
new file mode 100644
index 000000000000..b7c8d3a00c2c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
@@ -0,0 +1,297 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/cacheflush.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-smem.h"
+
+static struct reserved_mem *cam_reserved_smem;
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%d pages:%u\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+
+	return 0;
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+
+	if (ret)
+		dev_err(dev, "Unable to declare smem  memory:\n", ret);
+	else
+		dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+			 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+	of_reserved_mem_device_release(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int rc = 0;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	rc = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev->smem_dev = smem_dev->dev;
+
+	if (cam_reserved_smem) {
+		smem_dev->smem_base = cam_reserved_smem->base;
+		smem_dev->smem_size = cam_reserved_smem->size;
+		dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+			 __func__,
+			 smem_dev->dev,
+			 &smem_dev->smem_base,
+			 (smem_dev->smem_size / SZ_1M));
+		mtk_cam_reserved_drm_sg_init(smem_dev);
+		mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+	}
+
+	return 0;
+}
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: no-map are not supported\n");
+		return -EINVAL;
+	}
+
+	cam_reserved_smem = rmem;
+
+	pr_info("%s:created DMA memory pool at %pa, size %u MiB\n",
+		__func__, &rmem->base, (rmem->size / SZ_1M));
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam-smem",
+		       mtk_cam_smem_dma_setup);
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..44fb876c87d5
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, V2, 11/11] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-05-10  1:58   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-10  1:58 UTC (permalink / raw)
  To: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, Jungo Lin, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../mtk-isp/isp_50/cam/mtk_cam-smem-dev.c     | 297 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  28 ++
 2 files changed, 325 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
new file mode 100644
index 000000000000..b7c8d3a00c2c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem-dev.c
@@ -0,0 +1,297 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/cacheflush.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-smem.h"
+
+static struct reserved_mem *cam_reserved_smem;
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+	void *cpu_addr, dma_addr_t dma_addr,
+	size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%d pages:%u\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static int mtk_cam_smem_setup_dma_ops(struct device *dev,
+				      struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+
+	return 0;
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+
+	if (ret)
+		dev_err(dev, "Unable to declare smem  memory:\n", ret);
+	else
+		dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+			 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+	of_reserved_mem_device_release(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int rc = 0;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	rc = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev->smem_dev = smem_dev->dev;
+
+	if (cam_reserved_smem) {
+		smem_dev->smem_base = cam_reserved_smem->base;
+		smem_dev->smem_size = cam_reserved_smem->size;
+		dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+			 __func__,
+			 smem_dev->dev,
+			 &smem_dev->smem_base,
+			 (smem_dev->smem_size / SZ_1M));
+		mtk_cam_reserved_drm_sg_init(smem_dev);
+		mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+	}
+
+	return 0;
+}
+
+static int __init mtk_cam_smem_dma_setup(struct reserved_mem *rmem)
+{
+	unsigned long node = rmem->fdt_node;
+
+	if (of_get_flat_dt_prop(node, "reusable", NULL))
+		return -EINVAL;
+
+	if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
+		pr_err("Reserved memory: no-map are not supported\n");
+		return -EINVAL;
+	}
+
+	cam_reserved_smem = rmem;
+
+	pr_info("%s:created DMA memory pool at %pa, size %u MiB\n",
+		__func__, &rmem->base, (rmem->size / SZ_1M));
+	return 0;
+}
+
+RESERVEDMEM_OF_DECLARE(mtk_cam_smem,
+		       "mediatek,reserve-memory-cam-smem",
+		       mtk_cam_smem_dma_setup);
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..44fb876c87d5
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-05-10  1:57   ` [RFC,V2,06/11] " Jungo Lin
  (?)
@ 2019-05-13  8:35     ` Hans Verkuil
  -1 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:35 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hans.verkuil, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

On 5/10/19 3:57 AM, Jungo Lin wrote:
> Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.

These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 1db220da3bcc..b79046d2d812 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -711,6 +711,20 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
>  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
>  
> +/* Vendor specific - Mediatek ISP compressed formats */
> +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */

Are these all compressed formats? What sort of compression is used? Can software unpack it,
or this is meant to be fed to other mediatek hardware blocks?

> +
>  /* SDR formats - used only for Software Defined Radio devices */
>  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
>  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> @@ -732,6 +746,12 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
>  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
>  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> +/* Vendor specific - Mediatek ISP parameters for firmware */
> +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */

The documentation for these meta formats either needs to point to
freely available mediatek documentation (i.e. no NDA needed), or it
has to be documented in a header or in the pixelformat documentation.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-13  8:35     ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:35 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hans.verkuil, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

On 5/10/19 3:57 AM, Jungo Lin wrote:
> Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.

These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 1db220da3bcc..b79046d2d812 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -711,6 +711,20 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
>  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
>  
> +/* Vendor specific - Mediatek ISP compressed formats */
> +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */

Are these all compressed formats? What sort of compression is used? Can software unpack it,
or this is meant to be fed to other mediatek hardware blocks?

> +
>  /* SDR formats - used only for Software Defined Radio devices */
>  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
>  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> @@ -732,6 +746,12 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
>  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
>  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> +/* Vendor specific - Mediatek ISP parameters for firmware */
> +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */

The documentation for these meta formats either needs to point to
freely available mediatek documentation (i.e. no NDA needed), or it
has to be documented in a header or in the pixelformat documentation.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-13  8:35     ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:35 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hans.verkuil, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

On 5/10/19 3:57 AM, Jungo Lin wrote:
> Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.

These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 1db220da3bcc..b79046d2d812 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -711,6 +711,20 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
>  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
>  
> +/* Vendor specific - Mediatek ISP compressed formats */
> +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */

Are these all compressed formats? What sort of compression is used? Can software unpack it,
or this is meant to be fed to other mediatek hardware blocks?

> +
>  /* SDR formats - used only for Software Defined Radio devices */
>  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
>  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> @@ -732,6 +746,12 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
>  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
>  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> +/* Vendor specific - Mediatek ISP parameters for firmware */
> +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */

The documentation for these meta formats either needs to point to
freely available mediatek documentation (i.e. no NDA needed), or it
has to be documented in a header or in the pixelformat documentation.

Regards,

	Hans

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
  2019-05-10  1:58   ` Jungo Lin
  (?)
@ 2019-05-13  8:46     ` Hans Verkuil
  -1 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:46 UTC (permalink / raw)
  To: Jungo Lin, tfiga, Sakari Ailus, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

On 5/10/19 3:58 AM, Jungo Lin wrote:
> Reserved Mediatek ISP P1 private control number with 16.
> Moreover, add two private controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.
> 
> 2. V4L2_CID_PRIVATE_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default image path is pure raw.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
>  include/uapi/linux/v4l2-controls.h            |   4 +
>  3 files changed, 169 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> new file mode 100644
> index 000000000000..520adbe367ed
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */

Don't combine both SPDX and a license text. Just use the SPDX.

I see it being used elsewhere as well, so I won't repeat myself.

> +
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam.h"
> +
> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +	unsigned int width, height;
> +
> +	width = imgo_fmt->fmt.pix_mp.width;
> +	height = imgo_fmt->fmt.pix_mp.height;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> +		width, height);
> +
> +	ctrl->val = (width << 16) | height;
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> +		handle_ctrl_get_bin_info(ctrl);
> +		break;
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		handle_ctrl_get_raw_path(ctrl);
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		return handle_ctrl_set_raw_path(ctrl);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,

Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
that this is mediatek-specific. Same for the next control below.

> +	.name = "MTK CAM GET BIN INFO",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,

Don't mix width and height. I recommend splitting this into two controls.

Sakari might have an opinion on this as well.

> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> +	.name = "MTK CAM RAW PATH",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},

RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Although you have to describe in the header or here what that means.

Private controls should be well documented.

> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {
> +		v4l2_ctrl_handler_free(hdl);
> +		return hdl->error;
> +	}
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> new file mode 100644
> index 000000000000..74a6538c81ac
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> @@ -0,0 +1,32 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_CTRL_H__
> +#define __MTK_CAM_CTRL_H__
> +
> +#include <media/v4l2-ctrls.h>
> +
> +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> +#define V4L2_CID_PRIVATE_RAW_PATH \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)

These last two defines can be on a single line.

They need to be documented in the header.

> +
> +#define V4L2_CID_MTK_CAM_MAX	16
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl);
> +
> +#endif /* __MTK_CAM_CTRL_H__ */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 06479f2fb3ae..cbe8f5f7782b 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -192,6 +192,10 @@ enum v4l2_colorfx {
>   * We reserve 16 controls for this driver. */
>  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
>  
> +/* The base for the mediatek ISP Pass 1 driver controls */
> +/* We reserve 16 controls for this driver. */
> +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> +
>  /* MPEG-class control IDs */
>  /* The MPEG controls are applicable to all codec controls
>   * and the 'MPEG' part of the define is historical */
> 

Regards,

	Hans

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-13  8:46     ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:46 UTC (permalink / raw)
  To: Jungo Lin, tfiga, Sakari Ailus, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: linux-mediatek, linux-arm-kernel, linux-media, devicetree,
	srv_heupstream, Sean.Cheng, sj.huang, christie.yu, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, seraph.huang, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

On 5/10/19 3:58 AM, Jungo Lin wrote:
> Reserved Mediatek ISP P1 private control number with 16.
> Moreover, add two private controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.
> 
> 2. V4L2_CID_PRIVATE_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default image path is pure raw.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
>  include/uapi/linux/v4l2-controls.h            |   4 +
>  3 files changed, 169 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> new file mode 100644
> index 000000000000..520adbe367ed
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */

Don't combine both SPDX and a license text. Just use the SPDX.

I see it being used elsewhere as well, so I won't repeat myself.

> +
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam.h"
> +
> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +	unsigned int width, height;
> +
> +	width = imgo_fmt->fmt.pix_mp.width;
> +	height = imgo_fmt->fmt.pix_mp.height;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> +		width, height);
> +
> +	ctrl->val = (width << 16) | height;
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> +		handle_ctrl_get_bin_info(ctrl);
> +		break;
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		handle_ctrl_get_raw_path(ctrl);
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		return handle_ctrl_set_raw_path(ctrl);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,

Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
that this is mediatek-specific. Same for the next control below.

> +	.name = "MTK CAM GET BIN INFO",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,

Don't mix width and height. I recommend splitting this into two controls.

Sakari might have an opinion on this as well.

> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> +	.name = "MTK CAM RAW PATH",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},

RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Although you have to describe in the header or here what that means.

Private controls should be well documented.

> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {
> +		v4l2_ctrl_handler_free(hdl);
> +		return hdl->error;
> +	}
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> new file mode 100644
> index 000000000000..74a6538c81ac
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> @@ -0,0 +1,32 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_CTRL_H__
> +#define __MTK_CAM_CTRL_H__
> +
> +#include <media/v4l2-ctrls.h>
> +
> +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> +#define V4L2_CID_PRIVATE_RAW_PATH \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)

These last two defines can be on a single line.

They need to be documented in the header.

> +
> +#define V4L2_CID_MTK_CAM_MAX	16
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl);
> +
> +#endif /* __MTK_CAM_CTRL_H__ */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 06479f2fb3ae..cbe8f5f7782b 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -192,6 +192,10 @@ enum v4l2_colorfx {
>   * We reserve 16 controls for this driver. */
>  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
>  
> +/* The base for the mediatek ISP Pass 1 driver controls */
> +/* We reserve 16 controls for this driver. */
> +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> +
>  /* MPEG-class control IDs */
>  /* The MPEG controls are applicable to all codec controls
>   * and the 'MPEG' part of the define is historical */
> 

Regards,

	Hans

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-13  8:46     ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2019-05-13  8:46 UTC (permalink / raw)
  To: Jungo Lin, tfiga, Sakari Ailus, laurent.pinchart+renesas,
	matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

On 5/10/19 3:58 AM, Jungo Lin wrote:
> Reserved Mediatek ISP P1 private control number with 16.
> Moreover, add two private controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.
> 
> 2. V4L2_CID_PRIVATE_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default image path is pure raw.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
>  include/uapi/linux/v4l2-controls.h            |   4 +
>  3 files changed, 169 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> new file mode 100644
> index 000000000000..520adbe367ed
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */

Don't combine both SPDX and a license text. Just use the SPDX.

I see it being used elsewhere as well, so I won't repeat myself.

> +
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam.h"
> +
> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +	unsigned int width, height;
> +
> +	width = imgo_fmt->fmt.pix_mp.width;
> +	height = imgo_fmt->fmt.pix_mp.height;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> +		width, height);
> +
> +	ctrl->val = (width << 16) | height;
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> +		handle_ctrl_get_bin_info(ctrl);
> +		break;
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		handle_ctrl_get_raw_path(ctrl);
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_PRIVATE_RAW_PATH:
> +		return handle_ctrl_set_raw_path(ctrl);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,

Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
that this is mediatek-specific. Same for the next control below.

> +	.name = "MTK CAM GET BIN INFO",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,

Don't mix width and height. I recommend splitting this into two controls.

Sakari might have an opinion on this as well.

> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> +	.name = "MTK CAM RAW PATH",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},

RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Although you have to describe in the header or here what that means.

Private controls should be well documented.

> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {
> +		v4l2_ctrl_handler_free(hdl);
> +		return hdl->error;
> +	}
> +
> +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> new file mode 100644
> index 000000000000..74a6538c81ac
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> @@ -0,0 +1,32 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_CTRL_H__
> +#define __MTK_CAM_CTRL_H__
> +
> +#include <media/v4l2-ctrls.h>
> +
> +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> +#define V4L2_CID_PRIVATE_RAW_PATH \
> +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)

These last two defines can be on a single line.

They need to be documented in the header.

> +
> +#define V4L2_CID_MTK_CAM_MAX	16
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl);
> +
> +#endif /* __MTK_CAM_CTRL_H__ */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 06479f2fb3ae..cbe8f5f7782b 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -192,6 +192,10 @@ enum v4l2_colorfx {
>   * We reserve 16 controls for this driver. */
>  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
>  
> +/* The base for the mediatek ISP Pass 1 driver controls */
> +/* We reserve 16 controls for this driver. */
> +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> +
>  /* MPEG-class control IDs */
>  /* The MPEG controls are applicable to all codec controls
>   * and the 'MPEG' part of the define is historical */
> 

Regards,

	Hans

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
  2019-05-13  8:46     ` Hans Verkuil
  (?)
@ 2019-05-14  6:23         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-14  6:23 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw,
	Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w, suleiman-F7+t8E8rja9g9hUCZPvPmw,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	seraph.huang-NuS5LvNUpcJWk0Htik3J/w,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	zwisler-F7+t8E8rja9g9hUCZPvPmw, shik-F7+t8E8rja9g9hUCZPvPmw,
	yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, tfiga-F7+t8E8rja9g9hUCZPvPmw,
	christie.yu-NuS5LvNUpcJWk0Htik3J/w, Sakari Ailus

Hi Hans,

Thank you for your comments.

On Mon, 2019-05-13 at 10:46 +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 private control number with 16.
> > Moreover, add two private controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> > 
> > 2. V4L2_CID_PRIVATE_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default image path is pure raw.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
> >  include/uapi/linux/v4l2-controls.h            |   4 +
> >  3 files changed, 169 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > new file mode 100644
> > index 000000000000..520adbe367ed
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > @@ -0,0 +1,133 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> 
> Don't combine both SPDX and a license text. Just use the SPDX.
> 
> I see it being used elsewhere as well, so I won't repeat myself.
> 

Ok, we will revise the license declaration and only keep SPDX license
only as below in all files.

// SPDX-License-Identifier: GPL-2.0
//
// Copyright (c) 2019 MediaTek Inc.

> > +
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam.h"
> > +
> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +	unsigned int width, height;
> > +
> > +	width = imgo_fmt->fmt.pix_mp.width;
> > +	height = imgo_fmt->fmt.pix_mp.height;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> > +		width, height);
> > +
> > +	ctrl->val = (width << 16) | height;
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> > +		handle_ctrl_get_bin_info(ctrl);
> > +		break;
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		handle_ctrl_get_raw_path(ctrl);
> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		return handle_ctrl_set_raw_path(ctrl);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 

We will adopt your suggestion and revise these naming in the next patch.

> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 

Ok, we will split this control into different two controls for width &
height usage.  

> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.

Yes, you are right. We will rename this control to
V4L2_CID_MTK_PROCESSING_RAW and describes its usage in detail.

> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 

Ok, we will pay attenuation on this.
We will provide the detail information of these controls in next patch.

> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

Best regards,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-14  6:23         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-14  6:23 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: tfiga, Sakari Ailus, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi Hans,

Thank you for your comments.

On Mon, 2019-05-13 at 10:46 +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 private control number with 16.
> > Moreover, add two private controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> > 
> > 2. V4L2_CID_PRIVATE_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default image path is pure raw.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
> >  include/uapi/linux/v4l2-controls.h            |   4 +
> >  3 files changed, 169 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > new file mode 100644
> > index 000000000000..520adbe367ed
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > @@ -0,0 +1,133 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> 
> Don't combine both SPDX and a license text. Just use the SPDX.
> 
> I see it being used elsewhere as well, so I won't repeat myself.
> 

Ok, we will revise the license declaration and only keep SPDX license
only as below in all files.

// SPDX-License-Identifier: GPL-2.0
//
// Copyright (c) 2019 MediaTek Inc.

> > +
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam.h"
> > +
> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +	unsigned int width, height;
> > +
> > +	width = imgo_fmt->fmt.pix_mp.width;
> > +	height = imgo_fmt->fmt.pix_mp.height;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> > +		width, height);
> > +
> > +	ctrl->val = (width << 16) | height;
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> > +		handle_ctrl_get_bin_info(ctrl);
> > +		break;
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		handle_ctrl_get_raw_path(ctrl);
> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		return handle_ctrl_set_raw_path(ctrl);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 

We will adopt your suggestion and revise these naming in the next patch.

> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 

Ok, we will split this control into different two controls for width &
height usage.  

> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.

Yes, you are right. We will rename this control to
V4L2_CID_MTK_PROCESSING_RAW and describes its usage in detail.

> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 

Ok, we will pay attenuation on this.
We will provide the detail information of these controls in next patch.

> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

Best regards,

Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-05-14  6:23         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-14  6:23 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, frederic.chen, seraph.huang,
	linux-media, devicetree, zwisler, shik, yuzhao, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, sj.huang, tfiga, christie.yu, Sakari Ailus

Hi Hans,

Thank you for your comments.

On Mon, 2019-05-13 at 10:46 +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 private control number with 16.
> > Moreover, add two private controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_PRIVATE_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> > 
> > 2. V4L2_CID_PRIVATE_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default image path is pure raw.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 133 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  32 +++++
> >  include/uapi/linux/v4l2-controls.h            |   4 +
> >  3 files changed, 169 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > new file mode 100644
> > index 000000000000..520adbe367ed
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
> > @@ -0,0 +1,133 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> 
> Don't combine both SPDX and a license text. Just use the SPDX.
> 
> I see it being used elsewhere as well, so I won't repeat myself.
> 

Ok, we will revise the license declaration and only keep SPDX license
only as below in all files.

// SPDX-License-Identifier: GPL-2.0
//
// Copyright (c) 2019 MediaTek Inc.

> > +
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam.h"
> > +
> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	const unsigned int idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	struct v4l2_format *imgo_fmt = &cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +	unsigned int width, height;
> > +
> > +	width = imgo_fmt->fmt.pix_mp.width;
> > +	height = imgo_fmt->fmt.pix_mp.height;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d",
> > +		width, height);
> > +
> > +	ctrl->val = (width << 16) | height;
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = p1_dev->isp_ctx.isp_raw_path;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get raw path:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_raw_path(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = ctrl->val;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set raw path:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_GET_BIN_INFO:
> > +		handle_ctrl_get_bin_info(ctrl);
> > +		break;
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		handle_ctrl_get_raw_path(ctrl);
> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_PRIVATE_RAW_PATH:
> > +		return handle_ctrl_set_raw_path(ctrl);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 

We will adopt your suggestion and revise these naming in the next patch.

> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 

Ok, we will split this control into different two controls for width &
height usage.  

> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.

Yes, you are right. We will rename this control to
V4L2_CID_MTK_PROCESSING_RAW and describes its usage in detail.

> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 

Ok, we will pay attenuation on this.
We will provide the detail information of these controls in next patch.

> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

Best regards,

Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  2019-05-10  1:57   ` [RFC,V2,01/11] " Jungo Lin
  (?)
@ 2019-05-14 19:50     ` Rob Herring
  -1 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> This patch adds the binding for describing the reserved
> shared memory used to exchange ISP configuration and tuning
> data between the co-processor and Pass 1 (P1) unit of the
> camera ISP system on Mediatek SoCs.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
>  1 file changed, 42 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

See my comments on the other 2 camera related reserved-memory bindings.

Rob

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-14 19:50     ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> This patch adds the binding for describing the reserved
> shared memory used to exchange ISP configuration and tuning
> data between the co-processor and Pass 1 (P1) unit of the
> camera ISP system on Mediatek SoCs.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
>  1 file changed, 42 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

See my comments on the other 2 camera related reserved-memory bindings.

Rob

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-14 19:50     ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> This patch adds the binding for describing the reserved
> shared memory used to exchange ISP configuration and tuning
> data between the co-processor and Pass 1 (P1) unit of the
> camera ISP system on Mediatek SoCs.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
>  1 file changed, 42 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt

See my comments on the other 2 camera related reserved-memory bindings.

Rob

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
  2019-05-10  1:57   ` Jungo Lin
  (?)
@ 2019-05-14 19:54     ` Rob Herring
  -1 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit in
> Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> out from the sensor interface, applies ISP image effects from tuning
> data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
>  1 file changed, 92 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..759e55a5dfac
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,92 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Must contain an entry for each entry in reg-names.

Must list reg-names here and define the values. Though, I don't find 
cam1, cam2, cam3 to be too useful.

> +- interrupts: interrupt number to the cpu.
> +- iommus: shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.

How many entries?

> +- power-domains : a phandle to the power domain of this local arbiter.
> +- clocks: device clocks, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- mediatek,scp : the node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- memory-region : the reserved shared memory region between Pass 1 unit and
> +  system control processor.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp", "syscon";
> +		reg = <0 0x1a000000 0 0x1000>,
> +		      <0 0x1a003000 0 0x1000>,
> +		      <0 0x1a004000 0 0x2000>,
> +		      <0 0x1a006000 0 0x2000>;
> +		reg-names = "camisp",
> +		            "cam1",
> +		            "cam2",
> +		            "cam3";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam1",
> +				  "cam2",
> +				  "cam3";
> +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> +			 <&iommu M4U_PORT_CAM_LSCI1>,
> +			 <&iommu M4U_PORT_CAM_BPCI>;
> +		#clock-cells = <1>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		/* Camera CCF */
> +		clocks = <&camsys CLK_CAM_CAM>,
> +			 <&camsys CLK_CAM_CAMTG>;
> +		clock-names = "CAMSYS_CAM_CGPDN",
> +			      "CAMSYS_CAMTG_CGPDN";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		mediatek,scp = <&scp>;
> +		memory-region = <&cam_mem_reserved>;
> +	};
> +
> +Reserved memory specific DT entry (see reserved memory binding for more
> +information):
> +
> +Example:
> +SoC specific DT entry:
> +
> +	cam_mem_reserved: cam_mem_region {
> +		compatible = "mediatek,reserve-memory-cam-smem";
> +		no-map;
> +		size = <0 0x01400000>; / *20 MB share mem size */
> +		alignment = <0 0x1000>;
> +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> +	};
> +
> +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> +contain one 'port' child node with child 'endpoint' node. Please refer to
> +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> +
> +Example:
> +Board specific DT entry:

Don't split examples like this.

> +
> +	&camisp {
> +		port@0 {
> +			seninf_0: endpoint {
> +				remote-endpoint = <&seninf_core>;
> +			};
> +		};
> +	};
> +
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-14 19:54     ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit in
> Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> out from the sensor interface, applies ISP image effects from tuning
> data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
>  1 file changed, 92 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..759e55a5dfac
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,92 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Must contain an entry for each entry in reg-names.

Must list reg-names here and define the values. Though, I don't find 
cam1, cam2, cam3 to be too useful.

> +- interrupts: interrupt number to the cpu.
> +- iommus: shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.

How many entries?

> +- power-domains : a phandle to the power domain of this local arbiter.
> +- clocks: device clocks, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- mediatek,scp : the node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- memory-region : the reserved shared memory region between Pass 1 unit and
> +  system control processor.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp", "syscon";
> +		reg = <0 0x1a000000 0 0x1000>,
> +		      <0 0x1a003000 0 0x1000>,
> +		      <0 0x1a004000 0 0x2000>,
> +		      <0 0x1a006000 0 0x2000>;
> +		reg-names = "camisp",
> +		            "cam1",
> +		            "cam2",
> +		            "cam3";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam1",
> +				  "cam2",
> +				  "cam3";
> +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> +			 <&iommu M4U_PORT_CAM_LSCI1>,
> +			 <&iommu M4U_PORT_CAM_BPCI>;
> +		#clock-cells = <1>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		/* Camera CCF */
> +		clocks = <&camsys CLK_CAM_CAM>,
> +			 <&camsys CLK_CAM_CAMTG>;
> +		clock-names = "CAMSYS_CAM_CGPDN",
> +			      "CAMSYS_CAMTG_CGPDN";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		mediatek,scp = <&scp>;
> +		memory-region = <&cam_mem_reserved>;
> +	};
> +
> +Reserved memory specific DT entry (see reserved memory binding for more
> +information):
> +
> +Example:
> +SoC specific DT entry:
> +
> +	cam_mem_reserved: cam_mem_region {
> +		compatible = "mediatek,reserve-memory-cam-smem";
> +		no-map;
> +		size = <0 0x01400000>; / *20 MB share mem size */
> +		alignment = <0 0x1000>;
> +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> +	};
> +
> +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> +contain one 'port' child node with child 'endpoint' node. Please refer to
> +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> +
> +Example:
> +Board specific DT entry:

Don't split examples like this.

> +
> +	&camisp {
> +		port@0 {
> +			seninf_0: endpoint {
> +				remote-endpoint = <&seninf_core>;
> +			};
> +		};
> +	};
> +
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-14 19:54     ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-05-14 19:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit in
> Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> out from the sensor interface, applies ISP image effects from tuning
> data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
>  1 file changed, 92 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..759e55a5dfac
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,92 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Must contain an entry for each entry in reg-names.

Must list reg-names here and define the values. Though, I don't find 
cam1, cam2, cam3 to be too useful.

> +- interrupts: interrupt number to the cpu.
> +- iommus: shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.

How many entries?

> +- power-domains : a phandle to the power domain of this local arbiter.
> +- clocks: device clocks, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- mediatek,scp : the node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- memory-region : the reserved shared memory region between Pass 1 unit and
> +  system control processor.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp", "syscon";
> +		reg = <0 0x1a000000 0 0x1000>,
> +		      <0 0x1a003000 0 0x1000>,
> +		      <0 0x1a004000 0 0x2000>,
> +		      <0 0x1a006000 0 0x2000>;
> +		reg-names = "camisp",
> +		            "cam1",
> +		            "cam2",
> +		            "cam3";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam1",
> +				  "cam2",
> +				  "cam3";
> +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> +			 <&iommu M4U_PORT_CAM_LSCI1>,
> +			 <&iommu M4U_PORT_CAM_BPCI>;
> +		#clock-cells = <1>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		/* Camera CCF */
> +		clocks = <&camsys CLK_CAM_CAM>,
> +			 <&camsys CLK_CAM_CAMTG>;
> +		clock-names = "CAMSYS_CAM_CGPDN",
> +			      "CAMSYS_CAMTG_CGPDN";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		mediatek,scp = <&scp>;
> +		memory-region = <&cam_mem_reserved>;
> +	};
> +
> +Reserved memory specific DT entry (see reserved memory binding for more
> +information):
> +
> +Example:
> +SoC specific DT entry:
> +
> +	cam_mem_reserved: cam_mem_region {
> +		compatible = "mediatek,reserve-memory-cam-smem";
> +		no-map;
> +		size = <0 0x01400000>; / *20 MB share mem size */
> +		alignment = <0 0x1000>;
> +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> +	};
> +
> +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> +contain one 'port' child node with child 'endpoint' node. Please refer to
> +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> +
> +Example:
> +Board specific DT entry:

Don't split examples like this.

> +
> +	&camisp {
> +		port@0 {
> +			seninf_0: endpoint {
> +				remote-endpoint = <&seninf_core>;
> +			};
> +		};
> +	};
> +
> -- 
> 2.18.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-05-13  8:35     ` [RFC,V2,06/11] " Hans Verkuil
  (?)
@ 2019-05-15 12:49       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 12:49 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Hans:

Thanks for your comments.

On Mon, 2019-05-13 at 10:35 +0200, Hans Verkuil wrote:
> On 5/10/19 3:57 AM, Jungo Lin wrote:
> > Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> 
> These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.
> 

Ok, we will add these pixfmt-<something>.rst files in next patch to
explain these pixel formats.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 1db220da3bcc..b79046d2d812 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -711,6 +711,20 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
> >  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
> >  
> > +/* Vendor specific - Mediatek ISP compressed formats */
> > +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> > +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
> 
> Are these all compressed formats? What sort of compression is used? Can software unpack it,
> or this is meant to be fed to other mediatek hardware blocks?
> 

No, these are not compressed formats. These images could be unpacked by
software, not depended on Mediatek hardware blocks.

> > +
> >  /* SDR formats - used only for Software Defined Radio devices */
> >  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
> >  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> > @@ -732,6 +746,12 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
> >  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> >  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> > +/* Vendor specific - Mediatek ISP parameters for firmware */
> > +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> > +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> > +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> > +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> > +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
> 
> The documentation for these meta formats either needs to point to
> freely available mediatek documentation (i.e. no NDA needed), or it
> has to be documented in a header or in the pixelformat documentation.
> 

Ok, we are under internal discussion how to export these meta
information.

> Regards,
> 
> 	Hans
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-15 12:49       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 12:49 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu,
	srv_heupstream, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang,
	yuzhao, linux-mediatek, seraph.huang, zwisler, christie.yu,
	frederic.chen, linux-arm-kernel, linux-media

Hi, Hans:

Thanks for your comments.

On Mon, 2019-05-13 at 10:35 +0200, Hans Verkuil wrote:
> On 5/10/19 3:57 AM, Jungo Lin wrote:
> > Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> 
> These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.
> 

Ok, we will add these pixfmt-<something>.rst files in next patch to
explain these pixel formats.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 1db220da3bcc..b79046d2d812 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -711,6 +711,20 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
> >  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
> >  
> > +/* Vendor specific - Mediatek ISP compressed formats */
> > +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> > +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
> 
> Are these all compressed formats? What sort of compression is used? Can software unpack it,
> or this is meant to be fed to other mediatek hardware blocks?
> 

No, these are not compressed formats. These images could be unpacked by
software, not depended on Mediatek hardware blocks.

> > +
> >  /* SDR formats - used only for Software Defined Radio devices */
> >  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
> >  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> > @@ -732,6 +746,12 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
> >  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> >  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> > +/* Vendor specific - Mediatek ISP parameters for firmware */
> > +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> > +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> > +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> > +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> > +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
> 
> The documentation for these meta formats either needs to point to
> freely available mediatek documentation (i.e. no NDA needed), or it
> has to be documented in a header or in the pixelformat documentation.
> 

Ok, we are under internal discussion how to export these meta
information.

> Regards,
> 
> 	Hans
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-05-15 12:49       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 12:49 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Hans:

Thanks for your comments.

On Mon, 2019-05-13 at 10:35 +0200, Hans Verkuil wrote:
> On 5/10/19 3:57 AM, Jungo Lin wrote:
> > Add packed/unpacked/full-g bayer format with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> 
> These pixel formats will need to be documented in Documentation/media/uapi/v4l/pixfmt-<something>.rst.
> 

Ok, we will add these pixfmt-<something>.rst files in next patch to
explain these pixel formats.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  include/uapi/linux/videodev2.h | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 1db220da3bcc..b79046d2d812 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -711,6 +711,20 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
> >  #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
> >  
> > +/* Vendor specific - Mediatek ISP compressed formats */
> > +#define V4L2_PIX_FMT_MTISP_U8	v4l2_fourcc('M', 'T', 'U', '8') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U10  v4l2_fourcc('M', 'T', 'U', 'A') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U12  v4l2_fourcc('M', 'T', 'U', 'C') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_U14  v4l2_fourcc('M', 'T', 'U', 'E') /* Unpacked bayer format, 16-bit */
> > +#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_B10  v4l2_fourcc('M', 'T', 'B', 'A') /* Packed   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_B12  v4l2_fourcc('M', 'T', 'B', 'C') /* Packed   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_B14  v4l2_fourcc('M', 'T', 'B', 'E') /* Packed   bayer format, 14-bit */
> > +#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G   bayer format,  8-bit */
> > +#define V4L2_PIX_FMT_MTISP_F10  v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G   bayer format, 10-bit */
> > +#define V4L2_PIX_FMT_MTISP_F12  v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G   bayer format, 12-bit */
> > +#define V4L2_PIX_FMT_MTISP_F14  v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G   bayer format, 14-bit */
> 
> Are these all compressed formats? What sort of compression is used? Can software unpack it,
> or this is meant to be fed to other mediatek hardware blocks?
> 

No, these are not compressed formats. These images could be unpacked by
software, not depended on Mediatek hardware blocks.

> > +
> >  /* SDR formats - used only for Software Defined Radio devices */
> >  #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
> >  #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
> > @@ -732,6 +746,12 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
> >  #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> >  #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> > +/* Vendor specific - Mediatek ISP parameters for firmware */
> > +#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
> > +#define V4L2_META_FMT_MTISP_3A	   v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
> > +#define V4L2_META_FMT_MTISP_AF	   v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
> > +#define V4L2_META_FMT_MTISP_LCS	   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
> > +#define V4L2_META_FMT_MTISP_LMV	   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
> 
> The documentation for these meta formats either needs to point to
> freely available mediatek documentation (i.e. no NDA needed), or it
> has to be documented in a header or in the pixelformat documentation.
> 

Ok, we are under internal discussion how to export these meta
information.

> Regards,
> 
> 	Hans
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
  2019-05-14 19:50     ` Rob Herring
  (?)
@ 2019-05-15 13:02       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 13:02 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:50 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> > This patch adds the binding for describing the reserved
> > shared memory used to exchange ISP configuration and tuning
> > data between the co-processor and Pass 1 (P1) unit of the
> > camera ISP system on Mediatek SoCs.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
> >  1 file changed, 42 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
> 
> See my comments on the other 2 camera related reserved-memory bindings.
> 

Ok, we will align DIP & FD drivers's implementation.

Best regards,

Jungo

> Rob

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-15 13:02       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 13:02 UTC (permalink / raw)
  To: Rob Herring
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:50 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> > This patch adds the binding for describing the reserved
> > shared memory used to exchange ISP configuration and tuning
> > data between the co-processor and Pass 1 (P1) unit of the
> > camera ISP system on Mediatek SoCs.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
> >  1 file changed, 42 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
> 
> See my comments on the other 2 camera related reserved-memory bindings.
> 

Ok, we will align DIP & FD drivers's implementation.

Best regards,

Jungo

> Rob



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory
@ 2019-05-15 13:02       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-15 13:02 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:50 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:47AM +0800, Jungo Lin wrote:
> > This patch adds the binding for describing the reserved
> > shared memory used to exchange ISP configuration and tuning
> > data between the co-processor and Pass 1 (P1) unit of the
> > camera ISP system on Mediatek SoCs.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mediatek,reserve-memory-cam-smem.txt      | 42 +++++++++++++++++++
> >  1 file changed, 42 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-cam-smem.txt
> 
> See my comments on the other 2 camera related reserved-memory bindings.
> 

Ok, we will align DIP & FD drivers's implementation.

Best regards,

Jungo

> Rob



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
  2019-05-14 19:54     ` Rob Herring
  (?)
@ 2019-05-16  6:12       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-16  6:12 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, frankie.chiu, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:54 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit in
> > Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> > out from the sensor interface, applies ISP image effects from tuning
> > data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
> >  1 file changed, 92 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..759e55a5dfac
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,92 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Must contain an entry for each entry in reg-names.
> 
> Must list reg-names here and define the values. Though, I don't find 
> cam1, cam2, cam3 to be too useful.
> 

Ok, we will list all our supported reg-names in next patch.

> > +- interrupts: interrupt number to the cpu.
> > +- iommus: shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> 
> How many entries?
> 

Basic, we just need to add only one master port for IOMMU property.
We will revise this and drop the other two ports.

> > +- power-domains : a phandle to the power domain of this local arbiter.
> > +- clocks: device clocks, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> > +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- mediatek,scp : the node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- memory-region : the reserved shared memory region between Pass 1 unit and
> > +  system control processor.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp", "syscon";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +		      <0 0x1a003000 0 0x1000>,
> > +		      <0 0x1a004000 0 0x2000>,
> > +		      <0 0x1a006000 0 0x2000>;
> > +		reg-names = "camisp",
> > +		            "cam1",
> > +		            "cam2",
> > +		            "cam3";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam1",
> > +				  "cam2",
> > +				  "cam3";
> > +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> > +			 <&iommu M4U_PORT_CAM_LSCI1>,
> > +			 <&iommu M4U_PORT_CAM_BPCI>;
> > +		#clock-cells = <1>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		/* Camera CCF */
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +			 <&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "CAMSYS_CAM_CGPDN",
> > +			      "CAMSYS_CAMTG_CGPDN";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		mediatek,scp = <&scp>;
> > +		memory-region = <&cam_mem_reserved>;
> > +	};
> > +
> > +Reserved memory specific DT entry (see reserved memory binding for more
> > +information):
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	cam_mem_reserved: cam_mem_region {
> > +		compatible = "mediatek,reserve-memory-cam-smem";
> > +		no-map;
> > +		size = <0 0x01400000>; / *20 MB share mem size */
> > +		alignment = <0 0x1000>;
> > +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> > +	};
> > +
> > +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> > +contain one 'port' child node with child 'endpoint' node. Please refer to
> > +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> > +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> > +
> > +Example:
> > +Board specific DT entry:
> 
> Don't split examples like this.
> 

Ok, we will keep one example in next patch.

Best regards,

Jungo

> > +
> > +	&camisp {
> > +		port@0 {
> > +			seninf_0: endpoint {
> > +				remote-endpoint = <&seninf_core>;
> > +			};
> > +		};
> > +	};
> > +
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-16  6:12       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-16  6:12 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi, Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:54 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit in
> > Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> > out from the sensor interface, applies ISP image effects from tuning
> > data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
> >  1 file changed, 92 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..759e55a5dfac
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,92 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Must contain an entry for each entry in reg-names.
> 
> Must list reg-names here and define the values. Though, I don't find 
> cam1, cam2, cam3 to be too useful.
> 

Ok, we will list all our supported reg-names in next patch.

> > +- interrupts: interrupt number to the cpu.
> > +- iommus: shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> 
> How many entries?
> 

Basic, we just need to add only one master port for IOMMU property.
We will revise this and drop the other two ports.

> > +- power-domains : a phandle to the power domain of this local arbiter.
> > +- clocks: device clocks, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> > +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- mediatek,scp : the node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- memory-region : the reserved shared memory region between Pass 1 unit and
> > +  system control processor.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp", "syscon";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +		      <0 0x1a003000 0 0x1000>,
> > +		      <0 0x1a004000 0 0x2000>,
> > +		      <0 0x1a006000 0 0x2000>;
> > +		reg-names = "camisp",
> > +		            "cam1",
> > +		            "cam2",
> > +		            "cam3";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam1",
> > +				  "cam2",
> > +				  "cam3";
> > +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> > +			 <&iommu M4U_PORT_CAM_LSCI1>,
> > +			 <&iommu M4U_PORT_CAM_BPCI>;
> > +		#clock-cells = <1>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		/* Camera CCF */
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +			 <&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "CAMSYS_CAM_CGPDN",
> > +			      "CAMSYS_CAMTG_CGPDN";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		mediatek,scp = <&scp>;
> > +		memory-region = <&cam_mem_reserved>;
> > +	};
> > +
> > +Reserved memory specific DT entry (see reserved memory binding for more
> > +information):
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	cam_mem_reserved: cam_mem_region {
> > +		compatible = "mediatek,reserve-memory-cam-smem";
> > +		no-map;
> > +		size = <0 0x01400000>; / *20 MB share mem size */
> > +		alignment = <0 0x1000>;
> > +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> > +	};
> > +
> > +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> > +contain one 'port' child node with child 'endpoint' node. Please refer to
> > +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> > +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> > +
> > +Example:
> > +Board specific DT entry:
> 
> Don't split examples like this.
> 

Ok, we will keep one example in next patch.

Best regards,

Jungo

> > +
> > +	&camisp {
> > +		port@0 {
> > +			seninf_0: endpoint {
> > +				remote-endpoint = <&seninf_core>;
> > +			};
> > +		};
> > +	};
> > +
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-05-16  6:12       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-16  6:12 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, frankie.chiu, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Rob:

Thanks for your comments.

On Tue, 2019-05-14 at 14:54 -0500, Rob Herring wrote:
> On Fri, May 10, 2019 at 09:57:52AM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit in
> > Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
> > out from the sensor interface, applies ISP image effects from tuning
> > data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 92 +++++++++++++++++++
> >  1 file changed, 92 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..759e55a5dfac
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,92 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Must contain an entry for each entry in reg-names.
> 
> Must list reg-names here and define the values. Though, I don't find 
> cam1, cam2, cam3 to be too useful.
> 

Ok, we will list all our supported reg-names in next patch.

> > +- interrupts: interrupt number to the cpu.
> > +- iommus: shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> 
> How many entries?
> 

Basic, we just need to add only one master port for IOMMU property.
We will revise this and drop the other two ports.

> > +- power-domains : a phandle to the power domain of this local arbiter.
> > +- clocks: device clocks, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: must be "CAMSYS_CAM_CGPDN" and "CAMSYS_CAMTG_CGPDN".
> > +- mediatek,larb: must contain the local arbiters in the current SOCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- mediatek,scp : the node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- memory-region : the reserved shared memory region between Pass 1 unit and
> > +  system control processor.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp", "syscon";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +		      <0 0x1a003000 0 0x1000>,
> > +		      <0 0x1a004000 0 0x2000>,
> > +		      <0 0x1a006000 0 0x2000>;
> > +		reg-names = "camisp",
> > +		            "cam1",
> > +		            "cam2",
> > +		            "cam3";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +			     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam1",
> > +				  "cam2",
> > +				  "cam3";
> > +		iommus = <&iommu M4U_PORT_CAM_LSCI0>,
> > +			 <&iommu M4U_PORT_CAM_LSCI1>,
> > +			 <&iommu M4U_PORT_CAM_BPCI>;
> > +		#clock-cells = <1>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		/* Camera CCF */
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +			 <&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "CAMSYS_CAM_CGPDN",
> > +			      "CAMSYS_CAMTG_CGPDN";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		mediatek,scp = <&scp>;
> > +		memory-region = <&cam_mem_reserved>;
> > +	};
> > +
> > +Reserved memory specific DT entry (see reserved memory binding for more
> > +information):
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	cam_mem_reserved: cam_mem_region {
> > +		compatible = "mediatek,reserve-memory-cam-smem";
> > +		no-map;
> > +		size = <0 0x01400000>; / *20 MB share mem size */
> > +		alignment = <0 0x1000>;
> > +		alloc-ranges = <0 0x40000000 0 0x10000000>;
> > +	};
> > +
> > +Mediatek ISP P1 supports a single port node with MIPI-CSI2 bus. It should
> > +contain one 'port' child node with child 'endpoint' node. Please refer to
> > +the bindings defined in Documentation/devicetree/bindings/media/video-interfaces.txt
> > +and Documentation/devicetree/bindings/media/mediatek-seninf.txt.
> > +
> > +Example:
> > +Board specific DT entry:
> 
> Don't split examples like this.
> 

Ok, we will keep one example in next patch.

Best regards,

Jungo

> > +
> > +	&camisp {
> > +		port@0 {
> > +			seninf_0: endpoint {
> > +				remote-endpoint = <&seninf_core>;
> > +			};
> > +		};
> > +	};
> > +
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-05-10  1:58     ` Jungo Lin
  (?)
@ 2019-05-24 18:49       ` Drew Davenport
  -1 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 18:49 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Jungo,

On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and six video devices during
> initialization. Moreover, it also connects with sensor and
> senif drivers with V4L2 async APIs.

Thanks for the patch. I've made a few comments inline. As a general
comment, what do you think of merging mtk_cam-dev.c and
mtk_cam-v4l2-util.c into one file? They seem to call into one another
and I'm not sure how beneficial it is to have them separate.

I have some comments on the other patches in this series that came about
while I was reviewing this, which I will send as well.

[snip]
 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..5a581ab65945
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,19 @@
> +#
> +# Copyright (C) 2018 MediaTek Inc.
> +#
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +
> +mtk-cam-isp-objs += \
> +	mtk_cam.o mtk_cam-dev.o \
> +	mtk_cam-ctrl.o mtk_cam-scp.o \
> +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o

Some of these files are added in other patches. Consider adding files to
the Makefile in the same patch a file is added.

> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> new file mode 100644
> index 000000000000..dda8a7b161ee
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> @@ -0,0 +1,758 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + * Copyright (C) 2017 Google, Inc.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/of.h>
> +#include <linux/of_platform.h>
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-event.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-smem.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {

This is not used anywhere. Please remove.

> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> +static struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +static const struct mtk_cam_dev_queues_setting queues_setting = {
> +	.output_node_descs = output_queues,
> +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> +	.capture_node_descs = capture_queues,
> +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> +};

I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup

> +
> +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)

Since only one_pixel_mode is supported, this function can be removed and
the callsite replaced with ALIGN(size, 2). This function can be added
once more when other pixel modes are supported.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 8);
> +	case two_pixel_mode:
> +		return ALIGN(size, 4);
> +	case one_pixel_mode:
> +		return ALIGN(size, 2);
> +	default:
> +		break;
> +	}
> +	return 0;
> +}
> +
> +static unsigned int align_packetd_out_size(__u32 size,
> +					   unsigned int pix_mode,
> +					   __u32 fmt)

This is only ever called with one_pixel_mode. Remove the pix_mode
argument and unreachable code.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 16);
> +	case two_pixel_mode:
> +		return ALIGN(size, 8);
> +	case one_pixel_mode:
> +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> +			return ALIGN(size, 4);
> +		else
> +			return ALIGN(size, 8);
> +	default:
> +		return ALIGN(size, 16);
> +	}
> +	return 0;
> +}
> +
> +static __u32 cal_main_stream_stride(struct device *dev,
> +				    __u32 width,
> +				    __u32 pix_fmt,
> +				    __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +	/* expand stride, instead of shrink width */
> +	stride = align_main_stream_size(stride, pix_mode);
> +
> +	dev_dbg(dev,
> +		"main width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +	return stride;
> +}
> +
> +static __u32 cal_packed_out_stride(struct device *dev,
> +				   __u32 width,
> +				   __u32 pix_fmt,
> +				   __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = DIV_ROUND_UP(width * 3, 2);
> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +	/* expand stride, instead of shrink width */
> +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> +
> +	dev_dbg(dev,
> +		"packed width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 cal_img_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> +					     one_pixel_mode);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> +					    one_pixel_mode);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* The helper to configure the device context */
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting)

This is only ever called with the same mtk_cam_dev_queues_setting
struct. I think you can remove that struct altogether and just set the
mtk_cam_dev_node_desc* for each node from output_queues and
capture_queues directly.

Also this can be a static function.

> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < setting->total_output_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->output_node_descs[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < setting->total_capture_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->capture_node_descs[i];
> +
> +	cam_dev->dev_node_num = node_idx;

This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
that instead of dev_node_num?

> +}
> +
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *fram_param)
> +{
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +
> +	if (!cam_dev->streaming)
> +		return 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev,
> +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> +		fram_param->request_fd,
> +		fram_param->frame_seq_no,
> +		fram_param->state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> +		buf->vbb.sequence = fram_param->frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;
> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +
> +	return 0;
> +}
> +
> +/* Calcuate mplane pix format */
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;
> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = cal_img_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> +}
> +
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> +}
> +
> +/* Get the default format setting */
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue_desc,
> +				  struct v4l2_format *dest)
> +{
> +	struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_fmt_set_img(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);
> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +/* Get a free buffer from a video node */
> +static struct mtk_cam_dev_buffer *
> +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> +{
> +	struct mtk_cam_dev_buffer *buf;
> +	struct mtk_cam_video_device *vdev;
> +
> +	if (node > cam_dev->dev_node_num || node < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> +		return NULL;
> +	}
> +	vdev = &cam_dev->mem2mem2_nodes[node];
> +
> +	spin_lock(&vdev->slock);
> +	buf = list_first_entry_or_null(&vdev->pending_list,
> +				       struct mtk_cam_dev_buffer,
> +				       list);
> +	if (!buf) {
> +		spin_unlock(&vdev->slock);
> +		return NULL;
> +	}
> +	list_del(&buf->list);
> +	spin_unlock(&vdev->slock);

Can this be simplified by going:
spin_lock();
buf = list_first_entry_or_null(...);
if (buf) list_del(...);
spin_unlock();
return buf;

> +
> +	return buf;
> +}
> +
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)

This only ever returns 0, so make it a void function.

> +{
> +	unsigned int node;
> +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_dev_start_param s_param;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> +
> +	if (!cam_dev->streaming) {
> +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> +		return 0;
> +	}
> +
> +	/* Check all enabled nodes to collect its buffer  */
> +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> +			continue;
> +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> +		if (!buf)
> +			continue;
> +
> +		/* TBD: use buf_init callback function */
> +		buf->daddr =
> +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> +				cam_dev->smem_dev, buf->daddr);
> +		} else {
> +			buf->scp_addr = 0;
> +		}
> +
> +		dev_dbg(dev,
> +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> +			node,
> +			buf->vbb.request_fd,
> +			buf->vbb.vb2_buf.index,
> +			buf->vbb.vb2_buf.state,
> +			&buf->daddr,
> +			&buf->scp_addr);
> +
> +		s_param.buffers[node] = buf;
> +		s_param.request_fd = buf->vbb.request_fd;
> +	}
> +
> +	/* Trigger en-queued job to driver */
> +	mtk_isp_req_enqueue(dev, &s_param);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;
> +
> +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> +
> +	/* v4l2 sub-device registration */
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);
> +
> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret)

If this fails do we need to undo the stuff done in
mtk_cam_mem2mem2_v4l2_register?

> +		return ret;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> new file mode 100644
> index 000000000000..410460de44fa
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> @@ -0,0 +1,250 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#ifndef __MTK_CAM_DEV_H__
> +#define __MTK_CAM_DEV_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> +
> +#define MTK_CAM_DEV_NODES		11
> +
> +#define MTK_CAM_P1_META_IN_0		0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> +#define MTK_CAM_P1_META_OUT_0		3
> +#define MTK_CAM_P1_META_OUT_1		4
> +#define MTK_CAM_P1_META_OUT_2		5
> +#define MTK_CAM_P1_META_OUT_3		6
> +#define MTK_CAM_P1_TOTAL_CAPTURE	6

Please align macro values using tabs.

> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +};
> +
> +/* Attributes setup by device owner */
> +struct mtk_cam_dev_queues_setting {
> +	const struct mtk_cam_dev_node_desc *output_node_descs;
> +	unsigned int total_output_nodes;
> +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> +	unsigned int total_capture_nodes;
> +};
> +
> +struct mtk_cam_dev_start_param {
> +	int request_fd;
> +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> +};
> +
> +struct mtk_cam_dev_finish_param {
> +	int request_fd;
> +	unsigned int frame_seq_no;
> +	unsigned int state;
> +	struct list_head *list_buf;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	struct v4l2_format *fmts;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device
> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct video_device vdev;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */
> +	struct mutex lock;
> +	/* protect for pending_list */
> +	spinlock_t slock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @dev_node_num: The number of supported V4L2 video device nodes
> + * @request_fd:	The file descriptor of request API
> + * @request_count: The buffer count of request API
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->
> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int dev_node_num;
> +	int request_fd;
> +	unsigned int request_count;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler);
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting);
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *param);
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no);
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id);
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue,
> +				  struct v4l2_format *dest_fmt);
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id);
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_dev *
> +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> +{
> +	return container_of(__sd,
> +		struct mtk_cam_dev, subdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> +{
> +	return container_of(__vq,
> +		struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb,
> +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +#endif /* __MTK_CAM_DEV_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> new file mode 100644
> index 000000000000..196aaef3d854
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> @@ -0,0 +1,1086 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <media/v4l2-common.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define MTK_CAM_SENINF_PAD_SRC			4
> +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> +
> +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> +			       struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	cam_dev->request_fd = -1;
> +	cam_dev->request_count = 0;
> +
> +	return mtk_isp_open(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> +				struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	return mtk_isp_release(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)

"get" implies that this function will retrieve something without
side effects, which is not the case here. In the error case, the return
value is ignored by the caller as well.

Consider making this function return a struct v4l2_subdev* (or NULL in
the error case) and let the caller set mtk_cam_dev::sensor.

> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	cam_dev->sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count > 0) {
> +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!cam_dev->sensor) {
> +		dev_err(dev, "Sensor is not connected\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}
> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	/* Seninf must stream on first */
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_dev_queue_req_buffers(cam_dev);
> +	isp_composer_stream(isp_ctx, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	return -EPERM;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		goto fail_sensor_off;
> +	}
> +
> +	isp_composer_stream(isp_ctx, 0);
> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);

I'd be interested to get Tomasz's input here. If we fail to stream off
one of the subdevs, should we stream on the other one? What if that
fails? It's not clear to me the expectation when stream off fails, but
this seems a bit odd.

> +	return -EPERM;
> +}
> +
> +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> +				   int enable)

This can fit on one line.

> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> +					  struct v4l2_fh *fh,
> +					  struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_link_setup(struct media_entity *entity,
> +			      const struct media_pad *local,
> +	const struct media_pad *remote, u32 flags)

Strange indentation here.

> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> +		pad, remote->index);
> +
> +	if (pad < cam_dev->dev_node_num)
> +		cam_dev->mem2mem2_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> +		&buf->daddr, &buf->scp_addr);
> +
> +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	struct vb2_v4l2_buffer *v4l2_buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		v4l2_buf->request_fd,
> +		v4l2_buf->vb2_buf.index);
> +
> +	if (v4l2_buf->request_fd < 0) {
> +		mtk_cam_dev_queue_buffers(vb);
> +		return;
> +	}
> +
> +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> +		mtk_cam_dev->request_count =
> +			vb->req_obj.req->num_incomplete_objects;
> +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> +			v4l2_buf->request_fd,
> +			vb->req_obj.req->num_incomplete_objects);
> +	}
> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_cam_dev->request_count--;
> +
> +	if (!mtk_cam_dev->request_count) {
> +		mtk_cam_dev->request_fd = -1;
> +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> +	}
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count > 0)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +	else
> +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> +
> +	if (node->desc.smem_alloc) {
> +		alloc_devs[0] = cam_dev->smem_dev;
> +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> +	} else {
> +		alloc_devs[0] = &cam_dev->pdev->dev;
> +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> +			alloc_devs[0]);
> +	}
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Validate initialized num_planes & size[0] */
> +	if (*num_planes) {
> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	/* Initialize buffer queue & locks */
> +	INIT_LIST_HEAD(&node->pending_list);
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);

Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?

> +
> +	return 0;
> +}
> +
> +static bool
> +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> +			    struct mtk_cam_video_device *except)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +
> +		if (node == except)
> +			continue;
> +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> +			return false;
> +	}
> +
> +	return true;
> +}
> +
> +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +				       struct mtk_cam_video_device *node,
> +				       enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	unsigned int i;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	for (i = 0; i < node->vbq.num_buffers; ++i)
> +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(node->vbq.bufs[i], state);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> +			node->id);
> +		ret = -ENOLINK;
> +		goto fail_return_bufs;
> +	}
> +
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> +			node->id, __func__);
> +		goto fail_return_bufs;
> +	}
> +
> +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> +		return 0;
> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> +			node->id);
> +		goto fail_stop_pipeline;
> +	}
> +	return 0;
> +
> +fail_stop_pipeline:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_return_bufs:
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +
> +	/* Was this the first node with streaming disabled? */
> +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> +		/* Yes, really stop streaming now */
> +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +			dev_err(&cam_dev->pdev->dev,
> +				"failed to stop streaming\n");
> +	}
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> +			   struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct v4l2_format *dev_fmt;
> +	__u32  width, height;
> +
> +	if (in_fmt->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc,
> +					     in_fmt);
> +	}
> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);
> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);
> +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> +				       &in_fmt->fmt.pix_mp,
> +				       node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;
> +
> +	/* Get the valid format */
> +	mtk_cam_videoc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input)
> +{
> +	if (input->index > 0)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_CTRL:
> +		return v4l2_ctrl_subscribe_event(fh, sub);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->index > 0 || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +/* subdev internal operations */
> +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> +	.open = mtk_cam_subdev_open,
> +	.close = mtk_cam_subdev_close,
> +};
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream = mtk_cam_subdev_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {
> +	.link_setup = mtk_cam_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);
> +}

Move this function up with the other mtk_cam_vb2_* functions.

> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +/*
> + * Config node's video properties
> + * according to the device context requirement
> + */
> +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> +				 unsigned int node,
> +				 struct video_device *vdev,
> +				 struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev_node_desc *node_desc =
> +		&cam_dev->mem2mem2_nodes[node].desc;
> +
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> +	f->type = node_desc->buf_type;
> +	vdev->ioctl_ops = node_desc->ioctl_ops;
> +
> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     node_desc,
> +				     f);
> +}
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {
> +	.req_validate = vb2_request_validate,
> +	.req_queue = vb2_request_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	int ret;
> +
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, media_dev);
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler)

This can be a static function.

> +{
> +	int ret;
> +
> +	/* Set up v4l2 device */
> +	v4l2_dev->ctrl_handler = ctrl_handler;
> +	v4l2_dev->mdev = media_dev;
> +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> +	ret = v4l2_device_register(dev, v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);

The calling function will do this cleanup when this function fails, so
no need to do it here; just return ret.

> +
> +	return ret;
> +}
> +
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int num_nodes = cam_dev->dev_node_num;
> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		goto fail_media_dev;
> +	}
> +
> +	ret = mtk_cam_v4l2_register(dev,
> +				    &cam_dev->media_dev,
> +				    &cam_dev->v4l2_dev,
> +				    NULL);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);

It doesn't look like this gets free'd in any of the failure cases below.

> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> +
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < num_nodes; i++) {

Consider moving some of this loop to a new function. This would simplify
the failure handling below by by removing the fail_vdev and
fail_vdev_media_entity labels, whose cleanup could move into the helper
function. It would also break up this large function a bit.

> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +		struct video_device *vdev = &node->vdev;
> +		struct vb2_queue *vbq = &node->vbq;
> +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> +
> +		cam_dev->subdev_pads[i].flags = output ?
> +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +		/* Initialize miscellaneous variables */
> +		mutex_init(&node->lock);
> +		spin_lock_init(&node->slock);
> +		INIT_LIST_HEAD(&node->pending_list);
> +
> +		/* Initialize formats to default values */
> +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> +
> +		/* Initialize media entities */
> +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +		if (ret) {
> +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> +			goto fail_vdev_media_entity;
> +		}
> +		node->enabled = false;
> +		node->id = i;
> +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> +		vdev->entity.ops = NULL;
> +
> +		/* Initialize vbq */
> +		vbq->type = node->vdev_fmt.type;
> +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +			vbq->io_modes = VB2_MMAP;
> +		else
> +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +		if (node->desc.smem_alloc)
> +			vbq->bidirectional = 1;
> +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +			vdev->entity.function =
> +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +		vbq->ops = &mtk_cam_vb2_ops;
> +		vbq->mem_ops = &vb2_dma_contig_memops;
> +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +		/* Put the process hub sub device in the vb2 private data */
> +		vbq->drv_priv = cam_dev;
> +		vbq->lock = &node->lock;
> +		vbq->supports_requests = true;
> +
> +		ret = vb2_queue_init(vbq);
> +		if (ret) {
> +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Initialize vdev */
> +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +		vdev->release = video_device_release_empty;
> +		vdev->fops = &mtk_cam_v4l2_fops;
> +		vdev->lock = &node->lock;
> +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +		vdev->queue = &node->vbq;
> +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +		/* Enable private control for image video devices */
> +		if (node->desc.image) {
> +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +			vdev->ctrl_handler = &node->ctrl_handler;
> +		}
> +		video_set_drvdata(vdev, cam_dev);
> +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +		if (ret) {
> +			dev_err(dev, "failed to register vde:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Create link between video node and the subdev pad */
> +		if (output) {
> +			ret = media_create_pad_link(&vdev->entity, 0,
> +						    &cam_dev->subdev.entity,
> +						    i, link_flags);
> +		} else {
> +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> +						    i, &vdev->entity, 0,
> +						    link_flags);
> +		}
> +		if (ret)
> +			goto fail_link;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +	for (; i >= 0; i--) {
> +fail_link:
> +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> +fail_vdev:
> +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> +fail_vdev_media_entity:
> +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +fail_media_dev:
>
media_device_unregister and media_device_cleanup are called when
mtk_cam_media_register fails, so only need to call these under the
fail_v4l2_dev label.

> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		dev = &cam_dev->mem2mem2_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_SENINF_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_P1_HUB_PAD_SINK,
> +				    0);
> +	if (ret)

should this function return an error here?

> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);

Should anything be done to cam_dev-seninf here, such as setting it to
NULL?

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> +				    struct v4l2_fwnode_endpoint *vep,
> +				    struct v4l2_async_subdev *asd)
> +{
> +	return 0;
> +}
> +
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> +		&cam_dev->pdev->dev, &cam_dev->notifier,
> +		sizeof(struct v4l2_async_subdev),
> +		mtk_cam_dev_fwnode_parse);

As far as I can tell, the fourth argument is optional, so I think you
can just set NULL and remove mtk_cam_dev_fwnode_parse.

> +	if (ret < 0)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;
> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..73b36916da08
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,43 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Frederic Chen <frederic.chen@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap);
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes);
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_try_fmt(struct file *file,
> +			   void *fh, struct v4l2_format *in_fmt);
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input);
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f);
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub);
> +
> +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-24 18:49       ` Drew Davenport
  0 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 18:49 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi Jungo,

On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and six video devices during
> initialization. Moreover, it also connects with sensor and
> senif drivers with V4L2 async APIs.

Thanks for the patch. I've made a few comments inline. As a general
comment, what do you think of merging mtk_cam-dev.c and
mtk_cam-v4l2-util.c into one file? They seem to call into one another
and I'm not sure how beneficial it is to have them separate.

I have some comments on the other patches in this series that came about
while I was reviewing this, which I will send as well.

[snip]
 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..5a581ab65945
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,19 @@
> +#
> +# Copyright (C) 2018 MediaTek Inc.
> +#
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +
> +mtk-cam-isp-objs += \
> +	mtk_cam.o mtk_cam-dev.o \
> +	mtk_cam-ctrl.o mtk_cam-scp.o \
> +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o

Some of these files are added in other patches. Consider adding files to
the Makefile in the same patch a file is added.

> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> new file mode 100644
> index 000000000000..dda8a7b161ee
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> @@ -0,0 +1,758 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + * Copyright (C) 2017 Google, Inc.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/of.h>
> +#include <linux/of_platform.h>
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-event.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-smem.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {

This is not used anywhere. Please remove.

> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> +static struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +static const struct mtk_cam_dev_queues_setting queues_setting = {
> +	.output_node_descs = output_queues,
> +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> +	.capture_node_descs = capture_queues,
> +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> +};

I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup

> +
> +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)

Since only one_pixel_mode is supported, this function can be removed and
the callsite replaced with ALIGN(size, 2). This function can be added
once more when other pixel modes are supported.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 8);
> +	case two_pixel_mode:
> +		return ALIGN(size, 4);
> +	case one_pixel_mode:
> +		return ALIGN(size, 2);
> +	default:
> +		break;
> +	}
> +	return 0;
> +}
> +
> +static unsigned int align_packetd_out_size(__u32 size,
> +					   unsigned int pix_mode,
> +					   __u32 fmt)

This is only ever called with one_pixel_mode. Remove the pix_mode
argument and unreachable code.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 16);
> +	case two_pixel_mode:
> +		return ALIGN(size, 8);
> +	case one_pixel_mode:
> +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> +			return ALIGN(size, 4);
> +		else
> +			return ALIGN(size, 8);
> +	default:
> +		return ALIGN(size, 16);
> +	}
> +	return 0;
> +}
> +
> +static __u32 cal_main_stream_stride(struct device *dev,
> +				    __u32 width,
> +				    __u32 pix_fmt,
> +				    __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +	/* expand stride, instead of shrink width */
> +	stride = align_main_stream_size(stride, pix_mode);
> +
> +	dev_dbg(dev,
> +		"main width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +	return stride;
> +}
> +
> +static __u32 cal_packed_out_stride(struct device *dev,
> +				   __u32 width,
> +				   __u32 pix_fmt,
> +				   __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = DIV_ROUND_UP(width * 3, 2);
> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +	/* expand stride, instead of shrink width */
> +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> +
> +	dev_dbg(dev,
> +		"packed width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 cal_img_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> +					     one_pixel_mode);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> +					    one_pixel_mode);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* The helper to configure the device context */
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting)

This is only ever called with the same mtk_cam_dev_queues_setting
struct. I think you can remove that struct altogether and just set the
mtk_cam_dev_node_desc* for each node from output_queues and
capture_queues directly.

Also this can be a static function.

> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < setting->total_output_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->output_node_descs[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < setting->total_capture_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->capture_node_descs[i];
> +
> +	cam_dev->dev_node_num = node_idx;

This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
that instead of dev_node_num?

> +}
> +
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *fram_param)
> +{
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +
> +	if (!cam_dev->streaming)
> +		return 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev,
> +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> +		fram_param->request_fd,
> +		fram_param->frame_seq_no,
> +		fram_param->state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> +		buf->vbb.sequence = fram_param->frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;
> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +
> +	return 0;
> +}
> +
> +/* Calcuate mplane pix format */
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;
> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = cal_img_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> +}
> +
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> +}
> +
> +/* Get the default format setting */
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue_desc,
> +				  struct v4l2_format *dest)
> +{
> +	struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_fmt_set_img(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);
> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +/* Get a free buffer from a video node */
> +static struct mtk_cam_dev_buffer *
> +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> +{
> +	struct mtk_cam_dev_buffer *buf;
> +	struct mtk_cam_video_device *vdev;
> +
> +	if (node > cam_dev->dev_node_num || node < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> +		return NULL;
> +	}
> +	vdev = &cam_dev->mem2mem2_nodes[node];
> +
> +	spin_lock(&vdev->slock);
> +	buf = list_first_entry_or_null(&vdev->pending_list,
> +				       struct mtk_cam_dev_buffer,
> +				       list);
> +	if (!buf) {
> +		spin_unlock(&vdev->slock);
> +		return NULL;
> +	}
> +	list_del(&buf->list);
> +	spin_unlock(&vdev->slock);

Can this be simplified by going:
spin_lock();
buf = list_first_entry_or_null(...);
if (buf) list_del(...);
spin_unlock();
return buf;

> +
> +	return buf;
> +}
> +
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)

This only ever returns 0, so make it a void function.

> +{
> +	unsigned int node;
> +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_dev_start_param s_param;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> +
> +	if (!cam_dev->streaming) {
> +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> +		return 0;
> +	}
> +
> +	/* Check all enabled nodes to collect its buffer  */
> +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> +			continue;
> +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> +		if (!buf)
> +			continue;
> +
> +		/* TBD: use buf_init callback function */
> +		buf->daddr =
> +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> +				cam_dev->smem_dev, buf->daddr);
> +		} else {
> +			buf->scp_addr = 0;
> +		}
> +
> +		dev_dbg(dev,
> +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> +			node,
> +			buf->vbb.request_fd,
> +			buf->vbb.vb2_buf.index,
> +			buf->vbb.vb2_buf.state,
> +			&buf->daddr,
> +			&buf->scp_addr);
> +
> +		s_param.buffers[node] = buf;
> +		s_param.request_fd = buf->vbb.request_fd;
> +	}
> +
> +	/* Trigger en-queued job to driver */
> +	mtk_isp_req_enqueue(dev, &s_param);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;
> +
> +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> +
> +	/* v4l2 sub-device registration */
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);
> +
> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret)

If this fails do we need to undo the stuff done in
mtk_cam_mem2mem2_v4l2_register?

> +		return ret;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> new file mode 100644
> index 000000000000..410460de44fa
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> @@ -0,0 +1,250 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#ifndef __MTK_CAM_DEV_H__
> +#define __MTK_CAM_DEV_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> +
> +#define MTK_CAM_DEV_NODES		11
> +
> +#define MTK_CAM_P1_META_IN_0		0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> +#define MTK_CAM_P1_META_OUT_0		3
> +#define MTK_CAM_P1_META_OUT_1		4
> +#define MTK_CAM_P1_META_OUT_2		5
> +#define MTK_CAM_P1_META_OUT_3		6
> +#define MTK_CAM_P1_TOTAL_CAPTURE	6

Please align macro values using tabs.

> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +};
> +
> +/* Attributes setup by device owner */
> +struct mtk_cam_dev_queues_setting {
> +	const struct mtk_cam_dev_node_desc *output_node_descs;
> +	unsigned int total_output_nodes;
> +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> +	unsigned int total_capture_nodes;
> +};
> +
> +struct mtk_cam_dev_start_param {
> +	int request_fd;
> +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> +};
> +
> +struct mtk_cam_dev_finish_param {
> +	int request_fd;
> +	unsigned int frame_seq_no;
> +	unsigned int state;
> +	struct list_head *list_buf;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	struct v4l2_format *fmts;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device
> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct video_device vdev;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */
> +	struct mutex lock;
> +	/* protect for pending_list */
> +	spinlock_t slock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @dev_node_num: The number of supported V4L2 video device nodes
> + * @request_fd:	The file descriptor of request API
> + * @request_count: The buffer count of request API
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->
> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int dev_node_num;
> +	int request_fd;
> +	unsigned int request_count;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler);
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting);
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *param);
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no);
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id);
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue,
> +				  struct v4l2_format *dest_fmt);
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id);
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_dev *
> +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> +{
> +	return container_of(__sd,
> +		struct mtk_cam_dev, subdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> +{
> +	return container_of(__vq,
> +		struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb,
> +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +#endif /* __MTK_CAM_DEV_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> new file mode 100644
> index 000000000000..196aaef3d854
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> @@ -0,0 +1,1086 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <media/v4l2-common.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define MTK_CAM_SENINF_PAD_SRC			4
> +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> +
> +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> +			       struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	cam_dev->request_fd = -1;
> +	cam_dev->request_count = 0;
> +
> +	return mtk_isp_open(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> +				struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	return mtk_isp_release(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)

"get" implies that this function will retrieve something without
side effects, which is not the case here. In the error case, the return
value is ignored by the caller as well.

Consider making this function return a struct v4l2_subdev* (or NULL in
the error case) and let the caller set mtk_cam_dev::sensor.

> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	cam_dev->sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count > 0) {
> +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!cam_dev->sensor) {
> +		dev_err(dev, "Sensor is not connected\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}
> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	/* Seninf must stream on first */
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_dev_queue_req_buffers(cam_dev);
> +	isp_composer_stream(isp_ctx, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	return -EPERM;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		goto fail_sensor_off;
> +	}
> +
> +	isp_composer_stream(isp_ctx, 0);
> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);

I'd be interested to get Tomasz's input here. If we fail to stream off
one of the subdevs, should we stream on the other one? What if that
fails? It's not clear to me the expectation when stream off fails, but
this seems a bit odd.

> +	return -EPERM;
> +}
> +
> +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> +				   int enable)

This can fit on one line.

> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> +					  struct v4l2_fh *fh,
> +					  struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_link_setup(struct media_entity *entity,
> +			      const struct media_pad *local,
> +	const struct media_pad *remote, u32 flags)

Strange indentation here.

> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> +		pad, remote->index);
> +
> +	if (pad < cam_dev->dev_node_num)
> +		cam_dev->mem2mem2_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> +		&buf->daddr, &buf->scp_addr);
> +
> +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	struct vb2_v4l2_buffer *v4l2_buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		v4l2_buf->request_fd,
> +		v4l2_buf->vb2_buf.index);
> +
> +	if (v4l2_buf->request_fd < 0) {
> +		mtk_cam_dev_queue_buffers(vb);
> +		return;
> +	}
> +
> +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> +		mtk_cam_dev->request_count =
> +			vb->req_obj.req->num_incomplete_objects;
> +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> +			v4l2_buf->request_fd,
> +			vb->req_obj.req->num_incomplete_objects);
> +	}
> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_cam_dev->request_count--;
> +
> +	if (!mtk_cam_dev->request_count) {
> +		mtk_cam_dev->request_fd = -1;
> +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> +	}
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count > 0)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +	else
> +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> +
> +	if (node->desc.smem_alloc) {
> +		alloc_devs[0] = cam_dev->smem_dev;
> +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> +	} else {
> +		alloc_devs[0] = &cam_dev->pdev->dev;
> +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> +			alloc_devs[0]);
> +	}
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Validate initialized num_planes & size[0] */
> +	if (*num_planes) {
> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	/* Initialize buffer queue & locks */
> +	INIT_LIST_HEAD(&node->pending_list);
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);

Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?

> +
> +	return 0;
> +}
> +
> +static bool
> +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> +			    struct mtk_cam_video_device *except)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +
> +		if (node == except)
> +			continue;
> +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> +			return false;
> +	}
> +
> +	return true;
> +}
> +
> +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +				       struct mtk_cam_video_device *node,
> +				       enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	unsigned int i;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	for (i = 0; i < node->vbq.num_buffers; ++i)
> +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(node->vbq.bufs[i], state);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> +			node->id);
> +		ret = -ENOLINK;
> +		goto fail_return_bufs;
> +	}
> +
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> +			node->id, __func__);
> +		goto fail_return_bufs;
> +	}
> +
> +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> +		return 0;
> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> +			node->id);
> +		goto fail_stop_pipeline;
> +	}
> +	return 0;
> +
> +fail_stop_pipeline:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_return_bufs:
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +
> +	/* Was this the first node with streaming disabled? */
> +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> +		/* Yes, really stop streaming now */
> +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +			dev_err(&cam_dev->pdev->dev,
> +				"failed to stop streaming\n");
> +	}
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> +			   struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct v4l2_format *dev_fmt;
> +	__u32  width, height;
> +
> +	if (in_fmt->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc,
> +					     in_fmt);
> +	}
> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);
> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);
> +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> +				       &in_fmt->fmt.pix_mp,
> +				       node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;
> +
> +	/* Get the valid format */
> +	mtk_cam_videoc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input)
> +{
> +	if (input->index > 0)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_CTRL:
> +		return v4l2_ctrl_subscribe_event(fh, sub);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->index > 0 || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +/* subdev internal operations */
> +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> +	.open = mtk_cam_subdev_open,
> +	.close = mtk_cam_subdev_close,
> +};
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream = mtk_cam_subdev_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {
> +	.link_setup = mtk_cam_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);
> +}

Move this function up with the other mtk_cam_vb2_* functions.

> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +/*
> + * Config node's video properties
> + * according to the device context requirement
> + */
> +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> +				 unsigned int node,
> +				 struct video_device *vdev,
> +				 struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev_node_desc *node_desc =
> +		&cam_dev->mem2mem2_nodes[node].desc;
> +
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> +	f->type = node_desc->buf_type;
> +	vdev->ioctl_ops = node_desc->ioctl_ops;
> +
> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     node_desc,
> +				     f);
> +}
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {
> +	.req_validate = vb2_request_validate,
> +	.req_queue = vb2_request_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	int ret;
> +
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, media_dev);
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler)

This can be a static function.

> +{
> +	int ret;
> +
> +	/* Set up v4l2 device */
> +	v4l2_dev->ctrl_handler = ctrl_handler;
> +	v4l2_dev->mdev = media_dev;
> +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> +	ret = v4l2_device_register(dev, v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);

The calling function will do this cleanup when this function fails, so
no need to do it here; just return ret.

> +
> +	return ret;
> +}
> +
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int num_nodes = cam_dev->dev_node_num;
> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		goto fail_media_dev;
> +	}
> +
> +	ret = mtk_cam_v4l2_register(dev,
> +				    &cam_dev->media_dev,
> +				    &cam_dev->v4l2_dev,
> +				    NULL);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);

It doesn't look like this gets free'd in any of the failure cases below.

> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> +
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < num_nodes; i++) {

Consider moving some of this loop to a new function. This would simplify
the failure handling below by by removing the fail_vdev and
fail_vdev_media_entity labels, whose cleanup could move into the helper
function. It would also break up this large function a bit.

> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +		struct video_device *vdev = &node->vdev;
> +		struct vb2_queue *vbq = &node->vbq;
> +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> +
> +		cam_dev->subdev_pads[i].flags = output ?
> +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +		/* Initialize miscellaneous variables */
> +		mutex_init(&node->lock);
> +		spin_lock_init(&node->slock);
> +		INIT_LIST_HEAD(&node->pending_list);
> +
> +		/* Initialize formats to default values */
> +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> +
> +		/* Initialize media entities */
> +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +		if (ret) {
> +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> +			goto fail_vdev_media_entity;
> +		}
> +		node->enabled = false;
> +		node->id = i;
> +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> +		vdev->entity.ops = NULL;
> +
> +		/* Initialize vbq */
> +		vbq->type = node->vdev_fmt.type;
> +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +			vbq->io_modes = VB2_MMAP;
> +		else
> +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +		if (node->desc.smem_alloc)
> +			vbq->bidirectional = 1;
> +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +			vdev->entity.function =
> +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +		vbq->ops = &mtk_cam_vb2_ops;
> +		vbq->mem_ops = &vb2_dma_contig_memops;
> +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +		/* Put the process hub sub device in the vb2 private data */
> +		vbq->drv_priv = cam_dev;
> +		vbq->lock = &node->lock;
> +		vbq->supports_requests = true;
> +
> +		ret = vb2_queue_init(vbq);
> +		if (ret) {
> +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Initialize vdev */
> +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +		vdev->release = video_device_release_empty;
> +		vdev->fops = &mtk_cam_v4l2_fops;
> +		vdev->lock = &node->lock;
> +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +		vdev->queue = &node->vbq;
> +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +		/* Enable private control for image video devices */
> +		if (node->desc.image) {
> +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +			vdev->ctrl_handler = &node->ctrl_handler;
> +		}
> +		video_set_drvdata(vdev, cam_dev);
> +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +		if (ret) {
> +			dev_err(dev, "failed to register vde:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Create link between video node and the subdev pad */
> +		if (output) {
> +			ret = media_create_pad_link(&vdev->entity, 0,
> +						    &cam_dev->subdev.entity,
> +						    i, link_flags);
> +		} else {
> +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> +						    i, &vdev->entity, 0,
> +						    link_flags);
> +		}
> +		if (ret)
> +			goto fail_link;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +	for (; i >= 0; i--) {
> +fail_link:
> +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> +fail_vdev:
> +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> +fail_vdev_media_entity:
> +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +fail_media_dev:
>
media_device_unregister and media_device_cleanup are called when
mtk_cam_media_register fails, so only need to call these under the
fail_v4l2_dev label.

> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		dev = &cam_dev->mem2mem2_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_SENINF_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_P1_HUB_PAD_SINK,
> +				    0);
> +	if (ret)

should this function return an error here?

> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);

Should anything be done to cam_dev-seninf here, such as setting it to
NULL?

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> +				    struct v4l2_fwnode_endpoint *vep,
> +				    struct v4l2_async_subdev *asd)
> +{
> +	return 0;
> +}
> +
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> +		&cam_dev->pdev->dev, &cam_dev->notifier,
> +		sizeof(struct v4l2_async_subdev),
> +		mtk_cam_dev_fwnode_parse);

As far as I can tell, the fourth argument is optional, so I think you
can just set NULL and remove mtk_cam_dev_fwnode_parse.

> +	if (ret < 0)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;
> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..73b36916da08
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,43 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Frederic Chen <frederic.chen@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap);
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes);
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_try_fmt(struct file *file,
> +			   void *fh, struct v4l2_format *in_fmt);
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input);
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f);
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub);
> +
> +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-24 18:49       ` Drew Davenport
  0 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 18:49 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Jungo,

On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and six video devices during
> initialization. Moreover, it also connects with sensor and
> senif drivers with V4L2 async APIs.

Thanks for the patch. I've made a few comments inline. As a general
comment, what do you think of merging mtk_cam-dev.c and
mtk_cam-v4l2-util.c into one file? They seem to call into one another
and I'm not sure how beneficial it is to have them separate.

I have some comments on the other patches in this series that came about
while I was reviewing this, which I will send as well.

[snip]
 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..5a581ab65945
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,19 @@
> +#
> +# Copyright (C) 2018 MediaTek Inc.
> +#
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +
> +mtk-cam-isp-objs += \
> +	mtk_cam.o mtk_cam-dev.o \
> +	mtk_cam-ctrl.o mtk_cam-scp.o \
> +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o

Some of these files are added in other patches. Consider adding files to
the Makefile in the same patch a file is added.

> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> new file mode 100644
> index 000000000000..dda8a7b161ee
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> @@ -0,0 +1,758 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + * Copyright (C) 2017 Google, Inc.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/of.h>
> +#include <linux/of_platform.h>
> +#include <linux/platform_device.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-event.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-smem.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {

This is not used anywhere. Please remove.

> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> +	/* buffer queue management */
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_videoc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> +static struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +static const struct mtk_cam_dev_queues_setting queues_setting = {
> +	.output_node_descs = output_queues,
> +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> +	.capture_node_descs = capture_queues,
> +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> +};

I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup

> +
> +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)

Since only one_pixel_mode is supported, this function can be removed and
the callsite replaced with ALIGN(size, 2). This function can be added
once more when other pixel modes are supported.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 8);
> +	case two_pixel_mode:
> +		return ALIGN(size, 4);
> +	case one_pixel_mode:
> +		return ALIGN(size, 2);
> +	default:
> +		break;
> +	}
> +	return 0;
> +}
> +
> +static unsigned int align_packetd_out_size(__u32 size,
> +					   unsigned int pix_mode,
> +					   __u32 fmt)

This is only ever called with one_pixel_mode. Remove the pix_mode
argument and unreachable code.

> +{
> +	switch (pix_mode) {
> +	case default_pixel_mode:
> +	case four_pixel_mode:
> +		return ALIGN(size, 16);
> +	case two_pixel_mode:
> +		return ALIGN(size, 8);
> +	case one_pixel_mode:
> +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> +			return ALIGN(size, 4);
> +		else
> +			return ALIGN(size, 8);
> +	default:
> +		return ALIGN(size, 16);
> +	}
> +	return 0;
> +}
> +
> +static __u32 cal_main_stream_stride(struct device *dev,
> +				    __u32 width,
> +				    __u32 pix_fmt,
> +				    __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +	/* expand stride, instead of shrink width */
> +	stride = align_main_stream_size(stride, pix_mode);
> +
> +	dev_dbg(dev,
> +		"main width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +	return stride;
> +}
> +
> +static __u32 cal_packed_out_stride(struct device *dev,
> +				   __u32 width,
> +				   __u32 pix_fmt,
> +				   __u32 pix_mode)

This function is only called with one_pixel_mode. Remove the pix_mode
argument.

> +{
> +	__u32 stride;
> +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);
> +	stride = DIV_ROUND_UP(width * 3, 2);
> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +	/* expand stride, instead of shrink width */
> +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> +
> +	dev_dbg(dev,
> +		"packed width:%d, pix_mode:%d, stride:%d\n",
> +		width, pix_mode, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 cal_img_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> +					     one_pixel_mode);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> +					    one_pixel_mode);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* The helper to configure the device context */
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting)

This is only ever called with the same mtk_cam_dev_queues_setting
struct. I think you can remove that struct altogether and just set the
mtk_cam_dev_node_desc* for each node from output_queues and
capture_queues directly.

Also this can be a static function.

> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < setting->total_output_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->output_node_descs[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < setting->total_capture_nodes; i++)
> +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> +			setting->capture_node_descs[i];
> +
> +	cam_dev->dev_node_num = node_idx;

This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
that instead of dev_node_num?

> +}
> +
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *fram_param)
> +{
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +
> +	if (!cam_dev->streaming)
> +		return 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev,
> +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> +		fram_param->request_fd,
> +		fram_param->frame_seq_no,
> +		fram_param->state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> +		buf->vbb.sequence = fram_param->frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;
> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +
> +	return 0;
> +}
> +
> +/* Calcuate mplane pix format */
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;
> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = cal_img_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> +}
> +
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> +}
> +
> +/* Get the default format setting */
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue_desc,
> +				  struct v4l2_format *dest)
> +{
> +	struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_fmt_set_img(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);
> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +/* Get a free buffer from a video node */
> +static struct mtk_cam_dev_buffer *
> +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> +{
> +	struct mtk_cam_dev_buffer *buf;
> +	struct mtk_cam_video_device *vdev;
> +
> +	if (node > cam_dev->dev_node_num || node < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> +		return NULL;
> +	}
> +	vdev = &cam_dev->mem2mem2_nodes[node];
> +
> +	spin_lock(&vdev->slock);
> +	buf = list_first_entry_or_null(&vdev->pending_list,
> +				       struct mtk_cam_dev_buffer,
> +				       list);
> +	if (!buf) {
> +		spin_unlock(&vdev->slock);
> +		return NULL;
> +	}
> +	list_del(&buf->list);
> +	spin_unlock(&vdev->slock);

Can this be simplified by going:
spin_lock();
buf = list_first_entry_or_null(...);
if (buf) list_del(...);
spin_unlock();
return buf;

> +
> +	return buf;
> +}
> +
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)

This only ever returns 0, so make it a void function.

> +{
> +	unsigned int node;
> +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_dev_start_param s_param;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> +
> +	if (!cam_dev->streaming) {
> +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> +		return 0;
> +	}
> +
> +	/* Check all enabled nodes to collect its buffer  */
> +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> +			continue;
> +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> +		if (!buf)
> +			continue;
> +
> +		/* TBD: use buf_init callback function */
> +		buf->daddr =
> +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> +				cam_dev->smem_dev, buf->daddr);
> +		} else {
> +			buf->scp_addr = 0;
> +		}
> +
> +		dev_dbg(dev,
> +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> +			node,
> +			buf->vbb.request_fd,
> +			buf->vbb.vb2_buf.index,
> +			buf->vbb.vb2_buf.state,
> +			&buf->daddr,
> +			&buf->scp_addr);
> +
> +		s_param.buffers[node] = buf;
> +		s_param.request_fd = buf->vbb.request_fd;
> +	}
> +
> +	/* Trigger en-queued job to driver */
> +	mtk_isp_req_enqueue(dev, &s_param);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;
> +
> +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> +
> +	/* v4l2 sub-device registration */
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);
> +
> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret)

If this fails do we need to undo the stuff done in
mtk_cam_mem2mem2_v4l2_register?

> +		return ret;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	return 0;
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> new file mode 100644
> index 000000000000..410460de44fa
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> @@ -0,0 +1,250 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#ifndef __MTK_CAM_DEV_H__
> +#define __MTK_CAM_DEV_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> +
> +#define MTK_CAM_DEV_NODES		11
> +
> +#define MTK_CAM_P1_META_IN_0		0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> +#define MTK_CAM_P1_META_OUT_0		3
> +#define MTK_CAM_P1_META_OUT_1		4
> +#define MTK_CAM_P1_META_OUT_2		5
> +#define MTK_CAM_P1_META_OUT_3		6
> +#define MTK_CAM_P1_TOTAL_CAPTURE	6

Please align macro values using tabs.

> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +};
> +
> +/* Attributes setup by device owner */
> +struct mtk_cam_dev_queues_setting {
> +	const struct mtk_cam_dev_node_desc *output_node_descs;
> +	unsigned int total_output_nodes;
> +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> +	unsigned int total_capture_nodes;
> +};
> +
> +struct mtk_cam_dev_start_param {
> +	int request_fd;
> +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> +};
> +
> +struct mtk_cam_dev_finish_param {
> +	int request_fd;
> +	unsigned int frame_seq_no;
> +	unsigned int state;
> +	struct list_head *list_buf;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	struct v4l2_format *fmts;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device
> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct video_device vdev;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */
> +	struct mutex lock;
> +	/* protect for pending_list */
> +	spinlock_t slock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @dev_node_num: The number of supported V4L2 video device nodes
> + * @request_fd:	The file descriptor of request API
> + * @request_count: The buffer count of request API
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->
> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int dev_node_num;
> +	int request_fd;
> +	unsigned int request_count;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler);
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> +int mtk_cam_dev_release(struct platform_device *pdev,
> +			struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> +			     const struct mtk_cam_dev_queues_setting *setting);
> +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> +			   struct mtk_cam_dev_finish_param *param);
> +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				 __u32 frame_seq_no);
> +void mtk_cam_dev_fmt_set_img(struct device *dev,
> +			     struct v4l2_pix_format_mplane *dest_fmt,
> +			     struct v4l2_pix_format_mplane *src_fmt,
> +			     unsigned int node_id);
> +struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> +void mtk_cam_dev_load_default_fmt(struct device *dev,
> +				  struct mtk_cam_dev_node_desc *queue,
> +				  struct v4l2_format *dest_fmt);
> +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> +				    struct v4l2_pix_format_mplane *dest_fmt,
> +				    unsigned int node_id);
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_dev *
> +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> +{
> +	return container_of(__sd,
> +		struct mtk_cam_dev, subdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> +{
> +	return container_of(__vq,
> +		struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb,
> +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +#endif /* __MTK_CAM_DEV_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> new file mode 100644
> index 000000000000..196aaef3d854
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> @@ -0,0 +1,1086 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 Mediatek Corporation.
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <media/v4l2-common.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define MTK_CAM_SENINF_PAD_SRC			4
> +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> +
> +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> +			       struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	cam_dev->request_fd = -1;
> +	cam_dev->request_count = 0;
> +
> +	return mtk_isp_open(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> +				struct v4l2_subdev_fh *fh)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	return mtk_isp_release(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)

"get" implies that this function will retrieve something without
side effects, which is not the case here. In the error case, the return
value is ignored by the caller as well.

Consider making this function return a struct v4l2_subdev* (or NULL in
the error case) and let the caller set mtk_cam_dev::sensor.

> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	cam_dev->sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count > 0) {
> +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!cam_dev->sensor) {
> +		dev_err(dev, "Sensor is not connected\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}
> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;
> +
> +	/* Seninf must stream on first */
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_dev_queue_req_buffers(cam_dev);
> +	isp_composer_stream(isp_ctx, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	return -EPERM;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		goto fail_sensor_off;
> +	}
> +
> +	isp_composer_stream(isp_ctx, 0);
> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);

I'd be interested to get Tomasz's input here. If we fail to stream off
one of the subdevs, should we stream on the other one? What if that
fails? It's not clear to me the expectation when stream off fails, but
this seems a bit odd.

> +	return -EPERM;
> +}
> +
> +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> +				   int enable)

This can fit on one line.

> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> +					  struct v4l2_fh *fh,
> +					  struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_link_setup(struct media_entity *entity,
> +			      const struct media_pad *local,
> +	const struct media_pad *remote, u32 flags)

Strange indentation here.

> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> +		pad, remote->index);
> +
> +	if (pad < cam_dev->dev_node_num)
> +		cam_dev->mem2mem2_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> +		&buf->daddr, &buf->scp_addr);
> +
> +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	struct vb2_v4l2_buffer *v4l2_buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		v4l2_buf->request_fd,
> +		v4l2_buf->vb2_buf.index);
> +
> +	if (v4l2_buf->request_fd < 0) {
> +		mtk_cam_dev_queue_buffers(vb);
> +		return;
> +	}
> +
> +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> +		mtk_cam_dev->request_count =
> +			vb->req_obj.req->num_incomplete_objects;
> +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> +			v4l2_buf->request_fd,
> +			vb->req_obj.req->num_incomplete_objects);
> +	}
> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_cam_dev->request_count--;
> +
> +	if (!mtk_cam_dev->request_count) {
> +		mtk_cam_dev->request_fd = -1;
> +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> +	}
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count > 0)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +	else
> +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> +
> +	if (node->desc.smem_alloc) {
> +		alloc_devs[0] = cam_dev->smem_dev;
> +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> +	} else {
> +		alloc_devs[0] = &cam_dev->pdev->dev;
> +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> +			alloc_devs[0]);
> +	}
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Validate initialized num_planes & size[0] */
> +	if (*num_planes) {
> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	/* Initialize buffer queue & locks */
> +	INIT_LIST_HEAD(&node->pending_list);
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);

Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?

> +
> +	return 0;
> +}
> +
> +static bool
> +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> +			    struct mtk_cam_video_device *except)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +
> +		if (node == except)
> +			continue;
> +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> +			return false;
> +	}
> +
> +	return true;
> +}
> +
> +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +				       struct mtk_cam_video_device *node,
> +				       enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	unsigned int i;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	for (i = 0; i < node->vbq.num_buffers; ++i)
> +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> +			vb2_buffer_done(node->vbq.bufs[i], state);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> +			node->id);
> +		ret = -ENOLINK;
> +		goto fail_return_bufs;
> +	}
> +
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> +			node->id, __func__);
> +		goto fail_return_bufs;
> +	}
> +
> +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> +		return 0;
> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret < 0) {
> +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> +			node->id);
> +		goto fail_stop_pipeline;
> +	}
> +	return 0;
> +
> +fail_stop_pipeline:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_return_bufs:
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +
> +	/* Was this the first node with streaming disabled? */
> +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> +		/* Yes, really stop streaming now */
> +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +			dev_err(&cam_dev->pdev->dev,
> +				"failed to stop streaming\n");
> +	}
> +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> +			   struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct v4l2_format *dev_fmt;
> +	__u32  width, height;
> +
> +	if (in_fmt->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc,
> +					     in_fmt);
> +	}
> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);
> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);
> +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> +				       &in_fmt->fmt.pix_mp,
> +				       node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;
> +
> +	/* Get the valid format */
> +	mtk_cam_videoc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input)
> +{
> +	if (input->index > 0)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_CTRL:
> +		return v4l2_ctrl_subscribe_event(fh, sub);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> +		sizes->stepwise.step_height = 1;
> +		sizes->stepwise.step_width = 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->index > 0 || f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +
> +	return 0;
> +}
> +
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	/* Each node is dedicated to only one meta format */
> +	if (f->type != node->vbq.type)
> +		return -EINVAL;
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +/* subdev internal operations */
> +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> +	.open = mtk_cam_subdev_open,
> +	.close = mtk_cam_subdev_close,
> +};
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream = mtk_cam_subdev_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {
> +	.link_setup = mtk_cam_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);
> +}

Move this function up with the other mtk_cam_vb2_* functions.

> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +/*
> + * Config node's video properties
> + * according to the device context requirement
> + */
> +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> +				 unsigned int node,
> +				 struct video_device *vdev,
> +				 struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev_node_desc *node_desc =
> +		&cam_dev->mem2mem2_nodes[node].desc;
> +
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> +	f->type = node_desc->buf_type;
> +	vdev->ioctl_ops = node_desc->ioctl_ops;
> +
> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     node_desc,
> +				     f);
> +}
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {
> +	.req_validate = vb2_request_validate,
> +	.req_queue = vb2_request_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	int ret;
> +
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, media_dev);
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_register(struct device *dev,
> +			  struct media_device *media_dev,
> +			  struct v4l2_device *v4l2_dev,
> +			  struct v4l2_ctrl_handler *ctrl_handler)

This can be a static function.

> +{
> +	int ret;
> +
> +	/* Set up v4l2 device */
> +	v4l2_dev->ctrl_handler = ctrl_handler;
> +	v4l2_dev->mdev = media_dev;
> +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> +	ret = v4l2_device_register(dev, v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	return 0;
> +
> +fail_v4l2_dev:
> +	media_device_unregister(media_dev);
> +	media_device_cleanup(media_dev);

The calling function will do this cleanup when this function fails, so
no need to do it here; just return ret.

> +
> +	return ret;
> +}
> +
> +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int num_nodes = cam_dev->dev_node_num;
> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		goto fail_media_dev;
> +	}
> +
> +	ret = mtk_cam_v4l2_register(dev,
> +				    &cam_dev->media_dev,
> +				    &cam_dev->v4l2_dev,
> +				    NULL);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;
> +	}
> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);

It doesn't look like this gets free'd in any of the failure cases below.

> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> +
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < num_nodes; i++) {

Consider moving some of this loop to a new function. This would simplify
the failure handling below by by removing the fail_vdev and
fail_vdev_media_entity labels, whose cleanup could move into the helper
function. It would also break up this large function a bit.

> +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> +		struct video_device *vdev = &node->vdev;
> +		struct vb2_queue *vbq = &node->vbq;
> +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> +
> +		cam_dev->subdev_pads[i].flags = output ?
> +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +		/* Initialize miscellaneous variables */
> +		mutex_init(&node->lock);
> +		spin_lock_init(&node->slock);
> +		INIT_LIST_HEAD(&node->pending_list);
> +
> +		/* Initialize formats to default values */
> +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> +
> +		/* Initialize media entities */
> +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +		if (ret) {
> +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> +			goto fail_vdev_media_entity;
> +		}
> +		node->enabled = false;
> +		node->id = i;
> +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> +		vdev->entity.ops = NULL;
> +
> +		/* Initialize vbq */
> +		vbq->type = node->vdev_fmt.type;
> +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +			vbq->io_modes = VB2_MMAP;
> +		else
> +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +		if (node->desc.smem_alloc)
> +			vbq->bidirectional = 1;
> +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +			vdev->entity.function =
> +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +		vbq->ops = &mtk_cam_vb2_ops;
> +		vbq->mem_ops = &vb2_dma_contig_memops;
> +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +		/* Put the process hub sub device in the vb2 private data */
> +		vbq->drv_priv = cam_dev;
> +		vbq->lock = &node->lock;
> +		vbq->supports_requests = true;
> +
> +		ret = vb2_queue_init(vbq);
> +		if (ret) {
> +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Initialize vdev */
> +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +		vdev->release = video_device_release_empty;
> +		vdev->fops = &mtk_cam_v4l2_fops;
> +		vdev->lock = &node->lock;
> +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +		vdev->queue = &node->vbq;
> +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +		/* Enable private control for image video devices */
> +		if (node->desc.image) {
> +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +			vdev->ctrl_handler = &node->ctrl_handler;
> +		}
> +		video_set_drvdata(vdev, cam_dev);
> +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +		if (ret) {
> +			dev_err(dev, "failed to register vde:%d\n", ret);
> +			goto fail_vdev;
> +		}
> +
> +		/* Create link between video node and the subdev pad */
> +		if (output) {
> +			ret = media_create_pad_link(&vdev->entity, 0,
> +						    &cam_dev->subdev.entity,
> +						    i, link_flags);
> +		} else {
> +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> +						    i, &vdev->entity, 0,
> +						    link_flags);
> +		}
> +		if (ret)
> +			goto fail_link;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +	for (; i >= 0; i--) {
> +fail_link:
> +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> +fail_vdev:
> +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> +fail_vdev_media_entity:
> +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +fail_media_dev:
>
media_device_unregister and media_device_cleanup are called when
mtk_cam_media_register fails, so only need to call these under the
fail_v4l2_dev label.

> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;
> +
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		dev = &cam_dev->mem2mem2_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_SENINF_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_P1_HUB_PAD_SINK,
> +				    0);
> +	if (ret)

should this function return an error here?

> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);

Should anything be done to cam_dev-seninf here, such as setting it to
NULL?

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> +				    struct v4l2_fwnode_endpoint *vep,
> +				    struct v4l2_async_subdev *asd)
> +{
> +	return 0;
> +}
> +
> +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> +		&cam_dev->pdev->dev, &cam_dev->notifier,
> +		sizeof(struct v4l2_async_subdev),
> +		mtk_cam_dev_fwnode_parse);

As far as I can tell, the fourth argument is optional, so I think you
can just set NULL and remove mtk_cam_dev_fwnode_parse.

> +	if (ret < 0)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;
> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..73b36916da08
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,43 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Frederic Chen <frederic.chen@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> +			    struct v4l2_capability *cap);
> +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> +			    struct v4l2_frmsizeenum *sizes);
> +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> +			    struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> +int mtk_cam_videoc_try_fmt(struct file *file,
> +			   void *fh, struct v4l2_format *in_fmt);
> +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +			      struct v4l2_input *input);
> +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> +			     struct v4l2_fmtdesc *f);
> +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> +			      struct v4l2_format *f);
> +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> +				   const struct v4l2_event_subscription *sub);
> +
> +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> -- 
> 2.18.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 09/11] media: platform: Add Mediatek ISP P1 device driver
  2019-05-10  1:58   ` Jungo Lin
  (?)
@ 2019-05-24 21:19     ` Drew Davenport
  -1 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 21:19 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Jungo,

On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.

A few comments inline.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
>  3 files changed, 1655 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..342f0e0e9837
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT	BIT(0)
> +#define CMOS_EN_BIT	BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST	BIT(0)
> +#define HW_PASS1_DON_ST	BIT(11)
> +#define SOF_INT_ST	BIT(12)
> +#define SW_PASS1_DON_ST	BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST	BIT(4)
> +#define TG_GBERR_ST	BIT(5)
> +#define CQ_CODE_ERR_ST	BIT(6)
> +#define CQ_APB_ERR_ST	BIT(7)
> +#define CQ_VS_ERR_ST	BIT(8)
> +#define AMX_ERR_ST	BIT(15)
> +#define RMX_ERR_ST	BIT(16)
> +#define BMX_ERR_ST	BIT(17)
> +#define RRZO_ERR_ST	BIT(18)
> +#define AFO_ERR_ST	BIT(19)
> +#define IMGO_ERR_ST	BIT(20)
> +#define AAO_ERR_ST	BIT(21)
> +#define PSO_ERR_ST	BIT(22)
> +#define LCSO_ERR_ST	BIT(23)
> +#define BNR_ERR_ST	BIT(24)
> +#define LSCI_ERR_ST	BIT(25)
> +#define DMA_ERR_ST	BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST	BIT(4)
> +#define AFO_DONE_ST	BIT(5)
> +#define AAO_DONE_ST	BIT(7)
> +#define PSO_DONE_ST	BIT(14)

Please align the values using tabs here and elsewhere.

> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM	( \
> +			VS_INT_ST |\
> +			SOF_INT_ST |\
> +			HW_PASS1_DON_ST |\
> +			SW_PASS1_DON_ST)
> +
> +/* IRQ Warning Mask */
> +#define INT_ST_MASK_CAM_WARN	(\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				PSO_ERR_ST | \
> +				LCSO_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR	(\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				CQ_CODE_ERR_ST |\
> +				CQ_APB_ERR_ST |\
> +				CQ_VS_ERR_ST |\
> +				BNR_ERR_ST |\
> +				RMX_ERR_ST |\
> +				BMX_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM	(\
> +				SOF_INT_ST |\
> +				SW_PASS1_DON_ST |\
> +				VS_INT_ST |\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM	(\
> +			AFO_DONE_ST |\
> +			AAO_DONE_ST |\
> +			PSO_DONE_ST |\
> +			FLKO_DONE_ST)
> +
> +/* Status check */
> +#define REG_CTL_EN		0x0004
> +#define REG_CTL_DMA_EN		0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2		0x0018
> +#define REG_CTL_RAW_INT_EN	0x0020
> +#define REG_CTL_RAW_INT_STAT	0x0024
> +#define REG_CTL_RAW_INT2_STAT	0x0034
> +#define REG_CTL_RAW_INT3_STAT	0x00c4
> +#define REG_CTL_TWIN_STAT	0x0050
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_SEN_GRAB_PIX	0x0238
> +#define REG_TG_SEN_GRAB_LIN	0x023c
> +#define REG_TG_VF_CON		0x0234
> +#define REG_TG_SUB_PERIOD	0x02a4
> +
> +#define REG_IMGO_BASE_ADDR	0x1020
> +#define REG_RRZO_BASE_ADDR	0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT	0x1360
> +#define REG_RRZO_ERR_STAT	0x1364
> +#define REG_AAO_ERR_STAT	0x1368
> +#define REG_AFO_ERR_STAT	0x136c
> +#define REG_LCSO_ERR_STAT	0x1370
> +#define REG_UFEO_ERR_STAT	0x1374
> +#define REG_PDO_ERR_STAT	0x1378
> +#define REG_BPCI_ERR_STAT	0x137c
> +#define REG_LSCI_ERR_STAT	0x1384
> +#define REG_PDI_ERR_STAT	0x138c
> +#define REG_LMVO_ERR_STAT	0x1390
> +#define REG_FLKO_ERR_STAT	0x1394
> +#define REG_PSO_ERR_STAT	0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR	0x0198
> +#define REG_HW_FRAME_NUM	0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX	0x14dc
> +#define REG_META1_VB2_INDEX	0x151c
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..fc874ec8f7f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1206 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +/* list of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> +};
> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> +			   unsigned int request_fd,
> +			   unsigned int frame_seq_no,
> +			   struct list_head *list_buf,
> +			   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev_finish_param fram_param;
> +
> +	fram_param.list_buf = list_buf;
> +	fram_param.request_fd = request_fd;
> +	fram_param.frame_seq_no = frame_seq_no;
> +	fram_param.state = state;
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> +		fram_param.request_fd,
> +		fram_param.frame_seq_no);
> +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> +	struct vb2_buffer *vb;
> +	struct vb2_v4l2_buffer *vbb;
> +
> +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> +		return;
> +
> +	mutex_lock(vb2_queue->lock);
> +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> +		vbb = to_vb2_v4l2_buffer(vb);
> +		if (vbb->request_fd < 0 &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vbb->vb2_buf.timestamp = ktime_get_ns();
> +			vbb->sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +		}
> +	}
> +	mutex_unlock(vb2_queue->lock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)
> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		if (ret == ERESTARTSYS) {
> +			dev_err(dev, "interrupted by a signal!\n");
> +			continue;
> +		}
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & VS_INT_ST) {
> +			/* Notify specific HW events to user space */
> +			mtk_cam_dev_event_frame_sync(cam_dev,
> +						     event_data.frame_seq_no);
> +			dev_dbg(dev,
> +				"event IRQ:0x%x DMA:0x%x is sent\n",
> +				event_data.irq_status_mask,
> +				event_data.dma_status_mask);
> +		}
> +
> +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int cq_addr_index;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	if (cq_num > frame_num) {
> +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> +
> +		writel(base_addr +
> +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> +			cq_num, frame_num, cq_addr_index);
> +	} else {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +	}
> +
> +	isp_dev->sof_count += 1;
> +
> +	return cq_num;
> +}
> +
> +static int irq_handle_notify_event(struct isp_device *isp_dev,
> +				   unsigned int irqstatus,
> +				   unsigned int dmastatus)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	unsigned long flags;
> +	int i;
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> +		(irqstatus & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> +		(dmastatus & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +
> +	dev_dbg(isp_dev->dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irqstatus & INT_ST_MASK_CAM),
> +		(dmastatus & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +
> +	return 0;
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cardinalnum, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev->streaming)
> +		return IRQ_HANDLED;
> +
> +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (irqstatus == 0)
> +		return IRQ_HANDLED;
> +
> +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> +
> +	/* sof , done order check . */
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> +		dev_warn(dev,
> +			 "isp sof_don block, sof_cnt:%d\n",
> +			 isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue ready frame */
> +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +		isp_dev->current_frame = hw_frame_num;
> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irqstatus & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +
> +		if ((irqstatus & INT_ST_MASK_CAM) ||
> +		    (dmastatus & DMA_ST_MASK_CAM))
> +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +	}
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	if (irqstatus & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	if (irqstatus & SW_PASS1_DON_ST) {
> +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> +
> +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible
> +				(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	/* check ISP error status */
> +	if (errstatus) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x/0x%x\n",
> +			irqstatus, warnstatus, errstatus);
> +
> +		/* show DMA errors in detail */
> +		if (errstatus & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,
> +			'A' + cardinalnum,
> +			isp_dev->sof_count,
> +			irqstatus,
> +			dmastatus,
> +			hw_frame_num,
> +			cq_num);
> +	return IRQ_HANDLED;
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> +				      p1_dev->isp_clk.clk_list);
> +	if (ret < 0)
> +		goto clk_err;
> +	return 0;
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int ret;
> +	unsigned int i;
> +
> +	/* Allocate context */
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;
> +
> +	p1_dev->isp_devs =
> +		devm_kzalloc(dev,
> +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	p1_dev->cam_dev =
> +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	/* iomap registers */
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;
> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> +			 i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)
> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* reg & interrupts index is shifted with 1  */
> +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> +			if (isp_dev->irq > 0) {
> +				ret = devm_request_irq(dev, isp_dev->irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),
> +						       (void *)isp_dev);
> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						isp_dev->irq);
> +					return ret;
> +				}
> +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> +					 isp_dev->irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}
> +
> +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_clk.clk_list =
> +		devm_kcalloc(dev,
> +			     p1_dev->isp_clk.num_clks,
> +			     sizeof(*p1_dev->isp_clk.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_clk.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_clk.num_clks,
> +				p1_dev->isp_clk.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory\n");
> +		return ret;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int usercount, module;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* disable VF */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, The TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}
> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int module, usercount;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev->streaming) {
> +		isp_dev = &p1_dev->isp_devs[module];
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}
> +	return 0;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> +		p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret < 0) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		mutex_init(&isp_ctx->composer_tx_lock);
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +	return 0;
> +}
> +
> +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> +
> +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)

I think s/enable/enabled would be a clearer name for both the function
and local variable.

> +{
> +	unsigned int enable_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enable_dma_ports = 0;
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		if (cam_dev->mem2mem2_nodes[i].enabled)
> +			enable_dma_ports |=
> +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> +	}
> +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> +		__func__, enable_dma_ports);
> +
> +	return enable_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return raw_pxl_id_b;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return raw_pxl_id_gb;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return raw_pxl_id_gr;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return raw_pxl_id_r;
> +	default:
> +		return raw_pxl_id_b;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return img_fmt_bayer8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return img_fmt_bayer10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return img_fmt_bayer12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return img_fmt_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return img_fmt_bayer8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return img_fmt_fg_bayer8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return img_fmt_bayer10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return img_fmt_fg_bayer10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return img_fmt_bayer12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return img_fmt_fg_bayer12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return img_fmt_bayer14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return img_fmt_fg_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 10;
> +	}
> +}
> +
> +static void composer_deinit_done_cb(void *data)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +
> +	disable_sys_clock(p1_dev);
> +	/* Notify PM */
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_open(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> +	int ret;
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> +
> +	if (usercount == 1) {
> +		ret = isp_setup_scp_rproc(p1_dev);
> +		if (ret)
> +			goto scp_err;
> +
> +		/* ISP HW INIT */
> +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> +		/* Use pure RAW as default HW path */
> +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +		/* Check enabled DMAs which is configured by media setup */
> +		isp_ctx->enable_dma_ports =
> +			get_enable_dma_ports(p1_dev->cam_dev);
> +
> +		if (!isp_ctx->enable_dma_ports) {
> +			dev_dbg(dev, "No DMAs are enabled\n");
> +			ret = -EINVAL;
> +			goto scp_err;
> +		}
> +
> +		pm_runtime_get_sync(dev);
> +
> +		ret = isp_init_context(p1_dev);
> +		if (ret)
> +			goto ctx_err;
> +		ret = isp_composer_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +		ret = isp_composer_hw_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +
> +		isp_composer_meta_config(&p1_dev->isp_ctx,
> +					 isp_ctx->enable_dma_ports);
> +	}
> +
> +	return 0;
> +composer_err:
> +	isp_uninit_context(p1_dev);
> +ctx_err:
> +	pm_runtime_put_sync(dev);
> +scp_err:
> +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> +	return ret;
> +}
> +
> +int mtk_isp_release(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> +		isp_uninit_context(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_format;
> +	unsigned int sd_width, sd_height;
> +	unsigned int enable_dma_ports, idx;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enable_dma_ports = isp_ctx->enable_dma_ports;
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> +
> +	/* sensor config */
> +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor,
> +			       pad, get_fmt, NULL, &sd_format);
> +
> +	if (ret) {
> +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev,
> +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_format.format.width, sd_format.format.height,
> +		sd_format.format.code, sd_format.format.field,
> +		sd_format.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> +	/* support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;
> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_format.format.code);
> +	config_param.cfg_in_param.img_fmt =
> +		get_sensor_fmt(sd_format.format.code);
> +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> +	sd_width = sd_format.format.width;
> +	sd_height = sd_format.format.height;
> +
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;

The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
mem2mem2_nodes directly here and below.

> +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> +		struct v4l2_format *imgo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_param.cfg_main_param.bypass = 0;
> +
> +		config_param.cfg_main_param.output.img_fmt =
> +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.pixel_byte =
> +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.size.w =
> +			imgo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_main_param.output.size.h =
> +			imgo_fmt->fmt.pix_mp.height;
> +
> +		config_param.cfg_main_param.output.size.stride =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_main_param.output.size.xsize =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_main_param.output.crop.left = 0x0;
> +		config_param.cfg_main_param.output.crop.top = 0x0;
> +
> +		config_param.cfg_main_param.output.crop.width = sd_width;
> +		config_param.cfg_main_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> +			  "img out:%d:%d in:%d:%d",
> +			  imgo_fmt->fmt.pix_mp.width,
> +			  imgo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev,
> +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> +			config_param.cfg_main_param.output.pixel_byte,
> +			config_param.cfg_main_param.output.img_fmt,
> +			config_param.cfg_main_param.pure_raw);
> +		dev_dbg(dev,
> +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_main_param.output.size.w,
> +			config_param.cfg_main_param.output.size.h,
> +			config_param.cfg_main_param.output.size.stride,
> +			config_param.cfg_main_param.output.size.xsize,
> +			config_param.cfg_main_param.output.crop.width,
> +			config_param.cfg_main_param.output.crop.height);
> +	} else {
> +		config_param.cfg_main_param.bypass = 1;
> +	}
> +
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> +		struct v4l2_format *rrzo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_resize_param.bypass = 0;
> +		config_param.cfg_resize_param.output.img_fmt =
> +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.pixel_byte =
> +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.size.w =
> +			rrzo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_resize_param.output.size.h =
> +			rrzo_fmt->fmt.pix_mp.height;
> +		config_param.cfg_resize_param.output.size.stride =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_resize_param.output.size.xsize =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_resize_param.output.crop.left = 0x0;
> +		config_param.cfg_resize_param.output.crop.top = 0x0;
> +		config_param.cfg_resize_param.output.crop.width = sd_width;
> +		config_param.cfg_resize_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> +			  "rrz out:%d:%d in:%d:%d",
> +			  rrzo_fmt->fmt.pix_mp.width,
> +			  rrzo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> +			config_param.cfg_resize_param.output.pixel_byte,
> +			config_param.cfg_resize_param.output.img_fmt);
> +		dev_dbg(dev,
> +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_resize_param.output.size.w,
> +			config_param.cfg_resize_param.output.size.h,
> +			config_param.cfg_resize_param.output.size.stride,
> +			config_param.cfg_resize_param.output.size.xsize,
> +			config_param.cfg_resize_param.output.crop.width,
> +			config_param.cfg_resize_param.output.crop.height);
> +	} else {
> +		config_param.cfg_resize_param.bypass = 1;
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> +
> +	isp_composer_hw_config(isp_ctx, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +	return 0;
> +}
> +
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_port,
> +		    struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct mtk_cam_dev_buffer **bundle_buffers;
> +	unsigned int i, idx;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> +	memset(framejob, 0, sizeof(*framejob));
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	bundle_buffers = &frameparamsbase->buffers[0];
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> +	framejob->request_fd = frameparamsbase->request_fd;
> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	idx = MTK_CAM_P1_META_IN_0;
> +	if (bundle_buffers[idx]) {
> +		frameparams.tuning_addr.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.tuning_addr.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Image output */
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[0].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "main stream address iova:0x%x\n",
> +			frameparams.img_dma_buffers[0].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Resize output */
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[1].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "packed out address iova:0x%x\n",
> +			frameparams.img_dma_buffers[1].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Meta output DMAs */
> +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> +		idx = MTK_CAM_P1_META_OUT_0 + i;
> +		if (bundle_buffers[idx]) {
> +			frameparams.meta_addrs[i].iova =
> +			  bundle_buffers[idx]->daddr;
> +			frameparams.meta_addrs[i].scp_addr =
> +			  bundle_buffers[idx]->scp_addr;
> +			list_add_tail(&bundle_buffers[idx]->list,
> +				      &framejob->list_buf);
> +		} else {
> +			frameparams.meta_addrs[i].iova = 0;
> +			frameparams.meta_addrs[i].scp_addr = 0;
> +		}
> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		frameparamsbase->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6cf8bb4ba93a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,300 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-scp.h"
> +
> +#define CAM_A_MAX_WIDTH		3328U
> +#define CAM_A_MAX_HEIGHT	2496U
> +#define CAM_B_MAX_WIDTH		5376U
> +#define CAM_B_MAX_HEIGHT	4032U
> +
> +#define CAM_MIN_WIDTH		80U
> +#define CAM_MIN_HEIGHT		60U
> +
> +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_PDO		BIT(6)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_RSSO		BIT(9)
> +#define R_PSO		BIT(10)
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_DATA_BUF_SIZE		4
> +#define COMPOSRE_EVENT_BUF_SIZE		4
> +
> +#define CQ_ADDRESS_OFFSET		0x640
> +#define CQ_BUFFER_COUNT			3

Please align macro values using tabs.

> +
> +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +enum {
> +	img_fmt_unknown		= 0x0000,
> +	img_fmt_raw_start	= 0x2200,
> +	img_fmt_bayer8		= img_fmt_raw_start,
> +	img_fmt_bayer10,
> +	img_fmt_bayer12,
> +	img_fmt_bayer14,
> +	img_fmt_fg_bayer8,
> +	img_fmt_fg_bayer10,
> +	img_fmt_fg_bayer12,
> +	img_fmt_fg_bayer14,
> +};
> +
> +enum {
> +	raw_pxl_id_b   = 0,
> +	raw_pxl_id_gb,
> +	raw_pxl_id_gr,
> +	raw_pxl_id_r
> +};
> +
> +enum {
> +	default_pixel_mode = 0,
> +	one_pixel_mode,
> +	two_pixel_mode,
> +	four_pixel_mode,
> +	pixel_mode_num,
> +};
> +
> +enum mtk_isp_scp_ipi_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};
> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_ipi_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};
> +
> +struct isp_clk_struct {
> +	int num_clks;
> +	struct clk_bulk_data *clk_list;
> +};
> +
> +struct isp_device {
> +	struct device *dev;
> +	void __iomem *regs;
> +	int irq;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	struct mutex composer_tx_lock; /* isp composer work protection */
> +
> +	struct isp_thread composer_rx_thread;
> +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> +	atomic_t composer_evts_start;
> +	atomic_t composer_evts_end;
> +	spinlock_t composer_evts_lock; /* SCP events protection */
> +	/* increase after ipi */
> +	atomic_t ipi_occupied;
> +	/* increase after frame enqueue */
> +	atomic_t composing_frame;
> +	/* current composed frame id */
> +	atomic_t composed_frame_id;
> +
> +	struct isp_queue p1_enqueue_list;
> +
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> +
> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	struct sg_table sgtable;
> +
> +	/* increase after open, decrease when close */
> +	atomic_t isp_user_cnt;
> +	/* frame sequence number, increase per en-queue*/
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;
> +	unsigned int enable_dma_ports;
> +
> +	void (*composer_deinit_donecb)(void *isp_ctx);
> +
> +	struct list_head list;
> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +
> +	/* for SCP driver  */
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct isp_clk_struct isp_clk;

What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
separate structs? They are only ever used in isp_p1_device.

> +	struct mtk_cam_dev *cam_dev;
> +	struct isp_device *isp_devs;

Similarly, why are these allocated at runtime rather then just members
of the struct?

In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
afterwards the struct mtk_cam_dev and struct isp_devices are. There will
only ever be ISP_DEV_NODE_NUM isp_devices.

Could this be changed to:
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
?

> +};
> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> +}
> +
> +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			      unsigned int dma);
> +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			    struct p1_config_param *config_param);
> +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> +			    void (*donecb)(void *data));
> +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> +			  void *data,
> +			  enum mtk_isp_scp_ipi_type type);

These functions are declared here, but implemented in mtk_cam-scp.c.
Can the funtion declarations be moved to mtk_cam-scp.h?

> +
> +/**
> + * mtk_isp_open - open isp driver and initialize related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_open(struct device *dev);
> +
> +/**
> + * mtk_isp_release - release isp driver and related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_release(struct device *dev);
> +
> +/**
> + * mtk_isp_config - output image & meta data configuration.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_config(struct device *dev);
> +
> +/**
> + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> + *
> + * @dev:	isp device.
> + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> + *
> + */
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase);
> +
> +/**
> + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> + * for non-per-frame DMA.
> + *
> + * @dev:	isp device.
> + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> + *
> + */
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_idx,
> +		    struct mtk_cam_dev_buffer *buffer);
> +#endif /*__CAMERA_ISP_H*/
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-24 21:19     ` Drew Davenport
  0 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 21:19 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi Jungo,

On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.

A few comments inline.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
>  3 files changed, 1655 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..342f0e0e9837
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT	BIT(0)
> +#define CMOS_EN_BIT	BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST	BIT(0)
> +#define HW_PASS1_DON_ST	BIT(11)
> +#define SOF_INT_ST	BIT(12)
> +#define SW_PASS1_DON_ST	BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST	BIT(4)
> +#define TG_GBERR_ST	BIT(5)
> +#define CQ_CODE_ERR_ST	BIT(6)
> +#define CQ_APB_ERR_ST	BIT(7)
> +#define CQ_VS_ERR_ST	BIT(8)
> +#define AMX_ERR_ST	BIT(15)
> +#define RMX_ERR_ST	BIT(16)
> +#define BMX_ERR_ST	BIT(17)
> +#define RRZO_ERR_ST	BIT(18)
> +#define AFO_ERR_ST	BIT(19)
> +#define IMGO_ERR_ST	BIT(20)
> +#define AAO_ERR_ST	BIT(21)
> +#define PSO_ERR_ST	BIT(22)
> +#define LCSO_ERR_ST	BIT(23)
> +#define BNR_ERR_ST	BIT(24)
> +#define LSCI_ERR_ST	BIT(25)
> +#define DMA_ERR_ST	BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST	BIT(4)
> +#define AFO_DONE_ST	BIT(5)
> +#define AAO_DONE_ST	BIT(7)
> +#define PSO_DONE_ST	BIT(14)

Please align the values using tabs here and elsewhere.

> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM	( \
> +			VS_INT_ST |\
> +			SOF_INT_ST |\
> +			HW_PASS1_DON_ST |\
> +			SW_PASS1_DON_ST)
> +
> +/* IRQ Warning Mask */
> +#define INT_ST_MASK_CAM_WARN	(\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				PSO_ERR_ST | \
> +				LCSO_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR	(\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				CQ_CODE_ERR_ST |\
> +				CQ_APB_ERR_ST |\
> +				CQ_VS_ERR_ST |\
> +				BNR_ERR_ST |\
> +				RMX_ERR_ST |\
> +				BMX_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM	(\
> +				SOF_INT_ST |\
> +				SW_PASS1_DON_ST |\
> +				VS_INT_ST |\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM	(\
> +			AFO_DONE_ST |\
> +			AAO_DONE_ST |\
> +			PSO_DONE_ST |\
> +			FLKO_DONE_ST)
> +
> +/* Status check */
> +#define REG_CTL_EN		0x0004
> +#define REG_CTL_DMA_EN		0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2		0x0018
> +#define REG_CTL_RAW_INT_EN	0x0020
> +#define REG_CTL_RAW_INT_STAT	0x0024
> +#define REG_CTL_RAW_INT2_STAT	0x0034
> +#define REG_CTL_RAW_INT3_STAT	0x00c4
> +#define REG_CTL_TWIN_STAT	0x0050
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_SEN_GRAB_PIX	0x0238
> +#define REG_TG_SEN_GRAB_LIN	0x023c
> +#define REG_TG_VF_CON		0x0234
> +#define REG_TG_SUB_PERIOD	0x02a4
> +
> +#define REG_IMGO_BASE_ADDR	0x1020
> +#define REG_RRZO_BASE_ADDR	0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT	0x1360
> +#define REG_RRZO_ERR_STAT	0x1364
> +#define REG_AAO_ERR_STAT	0x1368
> +#define REG_AFO_ERR_STAT	0x136c
> +#define REG_LCSO_ERR_STAT	0x1370
> +#define REG_UFEO_ERR_STAT	0x1374
> +#define REG_PDO_ERR_STAT	0x1378
> +#define REG_BPCI_ERR_STAT	0x137c
> +#define REG_LSCI_ERR_STAT	0x1384
> +#define REG_PDI_ERR_STAT	0x138c
> +#define REG_LMVO_ERR_STAT	0x1390
> +#define REG_FLKO_ERR_STAT	0x1394
> +#define REG_PSO_ERR_STAT	0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR	0x0198
> +#define REG_HW_FRAME_NUM	0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX	0x14dc
> +#define REG_META1_VB2_INDEX	0x151c
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..fc874ec8f7f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1206 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +/* list of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> +};
> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> +			   unsigned int request_fd,
> +			   unsigned int frame_seq_no,
> +			   struct list_head *list_buf,
> +			   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev_finish_param fram_param;
> +
> +	fram_param.list_buf = list_buf;
> +	fram_param.request_fd = request_fd;
> +	fram_param.frame_seq_no = frame_seq_no;
> +	fram_param.state = state;
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> +		fram_param.request_fd,
> +		fram_param.frame_seq_no);
> +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> +	struct vb2_buffer *vb;
> +	struct vb2_v4l2_buffer *vbb;
> +
> +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> +		return;
> +
> +	mutex_lock(vb2_queue->lock);
> +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> +		vbb = to_vb2_v4l2_buffer(vb);
> +		if (vbb->request_fd < 0 &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vbb->vb2_buf.timestamp = ktime_get_ns();
> +			vbb->sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +		}
> +	}
> +	mutex_unlock(vb2_queue->lock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)
> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		if (ret == ERESTARTSYS) {
> +			dev_err(dev, "interrupted by a signal!\n");
> +			continue;
> +		}
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & VS_INT_ST) {
> +			/* Notify specific HW events to user space */
> +			mtk_cam_dev_event_frame_sync(cam_dev,
> +						     event_data.frame_seq_no);
> +			dev_dbg(dev,
> +				"event IRQ:0x%x DMA:0x%x is sent\n",
> +				event_data.irq_status_mask,
> +				event_data.dma_status_mask);
> +		}
> +
> +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int cq_addr_index;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	if (cq_num > frame_num) {
> +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> +
> +		writel(base_addr +
> +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> +			cq_num, frame_num, cq_addr_index);
> +	} else {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +	}
> +
> +	isp_dev->sof_count += 1;
> +
> +	return cq_num;
> +}
> +
> +static int irq_handle_notify_event(struct isp_device *isp_dev,
> +				   unsigned int irqstatus,
> +				   unsigned int dmastatus)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	unsigned long flags;
> +	int i;
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> +		(irqstatus & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> +		(dmastatus & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +
> +	dev_dbg(isp_dev->dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irqstatus & INT_ST_MASK_CAM),
> +		(dmastatus & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +
> +	return 0;
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cardinalnum, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev->streaming)
> +		return IRQ_HANDLED;
> +
> +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (irqstatus == 0)
> +		return IRQ_HANDLED;
> +
> +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> +
> +	/* sof , done order check . */
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> +		dev_warn(dev,
> +			 "isp sof_don block, sof_cnt:%d\n",
> +			 isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue ready frame */
> +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +		isp_dev->current_frame = hw_frame_num;
> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irqstatus & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +
> +		if ((irqstatus & INT_ST_MASK_CAM) ||
> +		    (dmastatus & DMA_ST_MASK_CAM))
> +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +	}
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	if (irqstatus & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	if (irqstatus & SW_PASS1_DON_ST) {
> +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> +
> +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible
> +				(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	/* check ISP error status */
> +	if (errstatus) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x/0x%x\n",
> +			irqstatus, warnstatus, errstatus);
> +
> +		/* show DMA errors in detail */
> +		if (errstatus & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,
> +			'A' + cardinalnum,
> +			isp_dev->sof_count,
> +			irqstatus,
> +			dmastatus,
> +			hw_frame_num,
> +			cq_num);
> +	return IRQ_HANDLED;
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> +				      p1_dev->isp_clk.clk_list);
> +	if (ret < 0)
> +		goto clk_err;
> +	return 0;
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int ret;
> +	unsigned int i;
> +
> +	/* Allocate context */
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;
> +
> +	p1_dev->isp_devs =
> +		devm_kzalloc(dev,
> +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	p1_dev->cam_dev =
> +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	/* iomap registers */
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;
> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> +			 i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)
> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* reg & interrupts index is shifted with 1  */
> +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> +			if (isp_dev->irq > 0) {
> +				ret = devm_request_irq(dev, isp_dev->irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),
> +						       (void *)isp_dev);
> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						isp_dev->irq);
> +					return ret;
> +				}
> +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> +					 isp_dev->irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}
> +
> +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_clk.clk_list =
> +		devm_kcalloc(dev,
> +			     p1_dev->isp_clk.num_clks,
> +			     sizeof(*p1_dev->isp_clk.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_clk.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_clk.num_clks,
> +				p1_dev->isp_clk.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory\n");
> +		return ret;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int usercount, module;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* disable VF */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, The TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}
> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int module, usercount;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev->streaming) {
> +		isp_dev = &p1_dev->isp_devs[module];
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}
> +	return 0;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> +		p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret < 0) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		mutex_init(&isp_ctx->composer_tx_lock);
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +	return 0;
> +}
> +
> +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> +
> +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)

I think s/enable/enabled would be a clearer name for both the function
and local variable.

> +{
> +	unsigned int enable_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enable_dma_ports = 0;
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		if (cam_dev->mem2mem2_nodes[i].enabled)
> +			enable_dma_ports |=
> +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> +	}
> +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> +		__func__, enable_dma_ports);
> +
> +	return enable_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return raw_pxl_id_b;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return raw_pxl_id_gb;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return raw_pxl_id_gr;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return raw_pxl_id_r;
> +	default:
> +		return raw_pxl_id_b;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return img_fmt_bayer8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return img_fmt_bayer10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return img_fmt_bayer12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return img_fmt_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return img_fmt_bayer8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return img_fmt_fg_bayer8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return img_fmt_bayer10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return img_fmt_fg_bayer10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return img_fmt_bayer12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return img_fmt_fg_bayer12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return img_fmt_bayer14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return img_fmt_fg_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 10;
> +	}
> +}
> +
> +static void composer_deinit_done_cb(void *data)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +
> +	disable_sys_clock(p1_dev);
> +	/* Notify PM */
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_open(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> +	int ret;
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> +
> +	if (usercount == 1) {
> +		ret = isp_setup_scp_rproc(p1_dev);
> +		if (ret)
> +			goto scp_err;
> +
> +		/* ISP HW INIT */
> +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> +		/* Use pure RAW as default HW path */
> +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +		/* Check enabled DMAs which is configured by media setup */
> +		isp_ctx->enable_dma_ports =
> +			get_enable_dma_ports(p1_dev->cam_dev);
> +
> +		if (!isp_ctx->enable_dma_ports) {
> +			dev_dbg(dev, "No DMAs are enabled\n");
> +			ret = -EINVAL;
> +			goto scp_err;
> +		}
> +
> +		pm_runtime_get_sync(dev);
> +
> +		ret = isp_init_context(p1_dev);
> +		if (ret)
> +			goto ctx_err;
> +		ret = isp_composer_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +		ret = isp_composer_hw_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +
> +		isp_composer_meta_config(&p1_dev->isp_ctx,
> +					 isp_ctx->enable_dma_ports);
> +	}
> +
> +	return 0;
> +composer_err:
> +	isp_uninit_context(p1_dev);
> +ctx_err:
> +	pm_runtime_put_sync(dev);
> +scp_err:
> +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> +	return ret;
> +}
> +
> +int mtk_isp_release(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> +		isp_uninit_context(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_format;
> +	unsigned int sd_width, sd_height;
> +	unsigned int enable_dma_ports, idx;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enable_dma_ports = isp_ctx->enable_dma_ports;
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> +
> +	/* sensor config */
> +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor,
> +			       pad, get_fmt, NULL, &sd_format);
> +
> +	if (ret) {
> +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev,
> +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_format.format.width, sd_format.format.height,
> +		sd_format.format.code, sd_format.format.field,
> +		sd_format.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> +	/* support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;
> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_format.format.code);
> +	config_param.cfg_in_param.img_fmt =
> +		get_sensor_fmt(sd_format.format.code);
> +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> +	sd_width = sd_format.format.width;
> +	sd_height = sd_format.format.height;
> +
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;

The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
mem2mem2_nodes directly here and below.

> +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> +		struct v4l2_format *imgo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_param.cfg_main_param.bypass = 0;
> +
> +		config_param.cfg_main_param.output.img_fmt =
> +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.pixel_byte =
> +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.size.w =
> +			imgo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_main_param.output.size.h =
> +			imgo_fmt->fmt.pix_mp.height;
> +
> +		config_param.cfg_main_param.output.size.stride =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_main_param.output.size.xsize =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_main_param.output.crop.left = 0x0;
> +		config_param.cfg_main_param.output.crop.top = 0x0;
> +
> +		config_param.cfg_main_param.output.crop.width = sd_width;
> +		config_param.cfg_main_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> +			  "img out:%d:%d in:%d:%d",
> +			  imgo_fmt->fmt.pix_mp.width,
> +			  imgo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev,
> +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> +			config_param.cfg_main_param.output.pixel_byte,
> +			config_param.cfg_main_param.output.img_fmt,
> +			config_param.cfg_main_param.pure_raw);
> +		dev_dbg(dev,
> +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_main_param.output.size.w,
> +			config_param.cfg_main_param.output.size.h,
> +			config_param.cfg_main_param.output.size.stride,
> +			config_param.cfg_main_param.output.size.xsize,
> +			config_param.cfg_main_param.output.crop.width,
> +			config_param.cfg_main_param.output.crop.height);
> +	} else {
> +		config_param.cfg_main_param.bypass = 1;
> +	}
> +
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> +		struct v4l2_format *rrzo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_resize_param.bypass = 0;
> +		config_param.cfg_resize_param.output.img_fmt =
> +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.pixel_byte =
> +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.size.w =
> +			rrzo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_resize_param.output.size.h =
> +			rrzo_fmt->fmt.pix_mp.height;
> +		config_param.cfg_resize_param.output.size.stride =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_resize_param.output.size.xsize =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_resize_param.output.crop.left = 0x0;
> +		config_param.cfg_resize_param.output.crop.top = 0x0;
> +		config_param.cfg_resize_param.output.crop.width = sd_width;
> +		config_param.cfg_resize_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> +			  "rrz out:%d:%d in:%d:%d",
> +			  rrzo_fmt->fmt.pix_mp.width,
> +			  rrzo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> +			config_param.cfg_resize_param.output.pixel_byte,
> +			config_param.cfg_resize_param.output.img_fmt);
> +		dev_dbg(dev,
> +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_resize_param.output.size.w,
> +			config_param.cfg_resize_param.output.size.h,
> +			config_param.cfg_resize_param.output.size.stride,
> +			config_param.cfg_resize_param.output.size.xsize,
> +			config_param.cfg_resize_param.output.crop.width,
> +			config_param.cfg_resize_param.output.crop.height);
> +	} else {
> +		config_param.cfg_resize_param.bypass = 1;
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> +
> +	isp_composer_hw_config(isp_ctx, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +	return 0;
> +}
> +
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_port,
> +		    struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct mtk_cam_dev_buffer **bundle_buffers;
> +	unsigned int i, idx;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> +	memset(framejob, 0, sizeof(*framejob));
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	bundle_buffers = &frameparamsbase->buffers[0];
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> +	framejob->request_fd = frameparamsbase->request_fd;
> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	idx = MTK_CAM_P1_META_IN_0;
> +	if (bundle_buffers[idx]) {
> +		frameparams.tuning_addr.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.tuning_addr.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Image output */
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[0].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "main stream address iova:0x%x\n",
> +			frameparams.img_dma_buffers[0].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Resize output */
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[1].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "packed out address iova:0x%x\n",
> +			frameparams.img_dma_buffers[1].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Meta output DMAs */
> +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> +		idx = MTK_CAM_P1_META_OUT_0 + i;
> +		if (bundle_buffers[idx]) {
> +			frameparams.meta_addrs[i].iova =
> +			  bundle_buffers[idx]->daddr;
> +			frameparams.meta_addrs[i].scp_addr =
> +			  bundle_buffers[idx]->scp_addr;
> +			list_add_tail(&bundle_buffers[idx]->list,
> +				      &framejob->list_buf);
> +		} else {
> +			frameparams.meta_addrs[i].iova = 0;
> +			frameparams.meta_addrs[i].scp_addr = 0;
> +		}
> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		frameparamsbase->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6cf8bb4ba93a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,300 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-scp.h"
> +
> +#define CAM_A_MAX_WIDTH		3328U
> +#define CAM_A_MAX_HEIGHT	2496U
> +#define CAM_B_MAX_WIDTH		5376U
> +#define CAM_B_MAX_HEIGHT	4032U
> +
> +#define CAM_MIN_WIDTH		80U
> +#define CAM_MIN_HEIGHT		60U
> +
> +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_PDO		BIT(6)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_RSSO		BIT(9)
> +#define R_PSO		BIT(10)
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_DATA_BUF_SIZE		4
> +#define COMPOSRE_EVENT_BUF_SIZE		4
> +
> +#define CQ_ADDRESS_OFFSET		0x640
> +#define CQ_BUFFER_COUNT			3

Please align macro values using tabs.

> +
> +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +enum {
> +	img_fmt_unknown		= 0x0000,
> +	img_fmt_raw_start	= 0x2200,
> +	img_fmt_bayer8		= img_fmt_raw_start,
> +	img_fmt_bayer10,
> +	img_fmt_bayer12,
> +	img_fmt_bayer14,
> +	img_fmt_fg_bayer8,
> +	img_fmt_fg_bayer10,
> +	img_fmt_fg_bayer12,
> +	img_fmt_fg_bayer14,
> +};
> +
> +enum {
> +	raw_pxl_id_b   = 0,
> +	raw_pxl_id_gb,
> +	raw_pxl_id_gr,
> +	raw_pxl_id_r
> +};
> +
> +enum {
> +	default_pixel_mode = 0,
> +	one_pixel_mode,
> +	two_pixel_mode,
> +	four_pixel_mode,
> +	pixel_mode_num,
> +};
> +
> +enum mtk_isp_scp_ipi_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};
> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_ipi_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};
> +
> +struct isp_clk_struct {
> +	int num_clks;
> +	struct clk_bulk_data *clk_list;
> +};
> +
> +struct isp_device {
> +	struct device *dev;
> +	void __iomem *regs;
> +	int irq;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	struct mutex composer_tx_lock; /* isp composer work protection */
> +
> +	struct isp_thread composer_rx_thread;
> +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> +	atomic_t composer_evts_start;
> +	atomic_t composer_evts_end;
> +	spinlock_t composer_evts_lock; /* SCP events protection */
> +	/* increase after ipi */
> +	atomic_t ipi_occupied;
> +	/* increase after frame enqueue */
> +	atomic_t composing_frame;
> +	/* current composed frame id */
> +	atomic_t composed_frame_id;
> +
> +	struct isp_queue p1_enqueue_list;
> +
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> +
> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	struct sg_table sgtable;
> +
> +	/* increase after open, decrease when close */
> +	atomic_t isp_user_cnt;
> +	/* frame sequence number, increase per en-queue*/
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;
> +	unsigned int enable_dma_ports;
> +
> +	void (*composer_deinit_donecb)(void *isp_ctx);
> +
> +	struct list_head list;
> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +
> +	/* for SCP driver  */
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct isp_clk_struct isp_clk;

What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
separate structs? They are only ever used in isp_p1_device.

> +	struct mtk_cam_dev *cam_dev;
> +	struct isp_device *isp_devs;

Similarly, why are these allocated at runtime rather then just members
of the struct?

In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
afterwards the struct mtk_cam_dev and struct isp_devices are. There will
only ever be ISP_DEV_NODE_NUM isp_devices.

Could this be changed to:
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
?

> +};
> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> +}
> +
> +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			      unsigned int dma);
> +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			    struct p1_config_param *config_param);
> +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> +			    void (*donecb)(void *data));
> +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> +			  void *data,
> +			  enum mtk_isp_scp_ipi_type type);

These functions are declared here, but implemented in mtk_cam-scp.c.
Can the funtion declarations be moved to mtk_cam-scp.h?

> +
> +/**
> + * mtk_isp_open - open isp driver and initialize related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_open(struct device *dev);
> +
> +/**
> + * mtk_isp_release - release isp driver and related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_release(struct device *dev);
> +
> +/**
> + * mtk_isp_config - output image & meta data configuration.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_config(struct device *dev);
> +
> +/**
> + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> + *
> + * @dev:	isp device.
> + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> + *
> + */
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase);
> +
> +/**
> + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> + * for non-per-frame DMA.
> + *
> + * @dev:	isp device.
> + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> + *
> + */
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_idx,
> +		    struct mtk_cam_dev_buffer *buffer);
> +#endif /*__CAMERA_ISP_H*/
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-24 21:19     ` Drew Davenport
  0 siblings, 0 replies; 388+ messages in thread
From: Drew Davenport @ 2019-05-24 21:19 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi Jungo,

On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.

A few comments inline.

> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
>  3 files changed, 1655 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..342f0e0e9837
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT	BIT(0)
> +#define CMOS_EN_BIT	BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST	BIT(0)
> +#define HW_PASS1_DON_ST	BIT(11)
> +#define SOF_INT_ST	BIT(12)
> +#define SW_PASS1_DON_ST	BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST	BIT(4)
> +#define TG_GBERR_ST	BIT(5)
> +#define CQ_CODE_ERR_ST	BIT(6)
> +#define CQ_APB_ERR_ST	BIT(7)
> +#define CQ_VS_ERR_ST	BIT(8)
> +#define AMX_ERR_ST	BIT(15)
> +#define RMX_ERR_ST	BIT(16)
> +#define BMX_ERR_ST	BIT(17)
> +#define RRZO_ERR_ST	BIT(18)
> +#define AFO_ERR_ST	BIT(19)
> +#define IMGO_ERR_ST	BIT(20)
> +#define AAO_ERR_ST	BIT(21)
> +#define PSO_ERR_ST	BIT(22)
> +#define LCSO_ERR_ST	BIT(23)
> +#define BNR_ERR_ST	BIT(24)
> +#define LSCI_ERR_ST	BIT(25)
> +#define DMA_ERR_ST	BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST	BIT(4)
> +#define AFO_DONE_ST	BIT(5)
> +#define AAO_DONE_ST	BIT(7)
> +#define PSO_DONE_ST	BIT(14)

Please align the values using tabs here and elsewhere.

> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM	( \
> +			VS_INT_ST |\
> +			SOF_INT_ST |\
> +			HW_PASS1_DON_ST |\
> +			SW_PASS1_DON_ST)
> +
> +/* IRQ Warning Mask */
> +#define INT_ST_MASK_CAM_WARN	(\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				PSO_ERR_ST | \
> +				LCSO_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR	(\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				CQ_CODE_ERR_ST |\
> +				CQ_APB_ERR_ST |\
> +				CQ_VS_ERR_ST |\
> +				BNR_ERR_ST |\
> +				RMX_ERR_ST |\
> +				BMX_ERR_ST |\
> +				BNR_ERR_ST |\
> +				LSCI_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM	(\
> +				SOF_INT_ST |\
> +				SW_PASS1_DON_ST |\
> +				VS_INT_ST |\
> +				TG_ERR_ST |\
> +				TG_GBERR_ST |\
> +				RRZO_ERR_ST |\
> +				AFO_ERR_ST |\
> +				IMGO_ERR_ST |\
> +				AAO_ERR_ST |\
> +				DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM	(\
> +			AFO_DONE_ST |\
> +			AAO_DONE_ST |\
> +			PSO_DONE_ST |\
> +			FLKO_DONE_ST)
> +
> +/* Status check */
> +#define REG_CTL_EN		0x0004
> +#define REG_CTL_DMA_EN		0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2		0x0018
> +#define REG_CTL_RAW_INT_EN	0x0020
> +#define REG_CTL_RAW_INT_STAT	0x0024
> +#define REG_CTL_RAW_INT2_STAT	0x0034
> +#define REG_CTL_RAW_INT3_STAT	0x00c4
> +#define REG_CTL_TWIN_STAT	0x0050
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_SEN_GRAB_PIX	0x0238
> +#define REG_TG_SEN_GRAB_LIN	0x023c
> +#define REG_TG_VF_CON		0x0234
> +#define REG_TG_SUB_PERIOD	0x02a4
> +
> +#define REG_IMGO_BASE_ADDR	0x1020
> +#define REG_RRZO_BASE_ADDR	0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT	0x1360
> +#define REG_RRZO_ERR_STAT	0x1364
> +#define REG_AAO_ERR_STAT	0x1368
> +#define REG_AFO_ERR_STAT	0x136c
> +#define REG_LCSO_ERR_STAT	0x1370
> +#define REG_UFEO_ERR_STAT	0x1374
> +#define REG_PDO_ERR_STAT	0x1378
> +#define REG_BPCI_ERR_STAT	0x137c
> +#define REG_LSCI_ERR_STAT	0x1384
> +#define REG_PDI_ERR_STAT	0x138c
> +#define REG_LMVO_ERR_STAT	0x1390
> +#define REG_FLKO_ERR_STAT	0x1394
> +#define REG_PSO_ERR_STAT	0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR	0x0198
> +#define REG_HW_FRAME_NUM	0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX	0x14dc
> +#define REG_META1_VB2_INDEX	0x151c
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..fc874ec8f7f0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1206 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +/* list of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> +};
> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> +			   unsigned int request_fd,
> +			   unsigned int frame_seq_no,
> +			   struct list_head *list_buf,
> +			   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev_finish_param fram_param;
> +
> +	fram_param.list_buf = list_buf;
> +	fram_param.request_fd = request_fd;
> +	fram_param.frame_seq_no = frame_seq_no;
> +	fram_param.state = state;
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> +		fram_param.request_fd,
> +		fram_param.frame_seq_no);
> +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> +	struct vb2_buffer *vb;
> +	struct vb2_v4l2_buffer *vbb;
> +
> +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> +		return;
> +
> +	mutex_lock(vb2_queue->lock);
> +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> +		vbb = to_vb2_v4l2_buffer(vb);
> +		if (vbb->request_fd < 0 &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vbb->vb2_buf.timestamp = ktime_get_ns();
> +			vbb->sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +		}
> +	}
> +	mutex_unlock(vb2_queue->lock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_isp_notify(isp_ctx,
> +				       framejob->request_fd,
> +				       framejob->frame_seq_no,
> +				       &framejob->list_buf,
> +				       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)
> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		if (ret == ERESTARTSYS) {
> +			dev_err(dev, "interrupted by a signal!\n");
> +			continue;
> +		}
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & VS_INT_ST) {
> +			/* Notify specific HW events to user space */
> +			mtk_cam_dev_event_frame_sync(cam_dev,
> +						     event_data.frame_seq_no);
> +			dev_dbg(dev,
> +				"event IRQ:0x%x DMA:0x%x is sent\n",
> +				event_data.irq_status_mask,
> +				event_data.dma_status_mask);
> +		}
> +
> +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int cq_addr_index;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	if (cq_num > frame_num) {
> +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> +
> +		writel(base_addr +
> +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> +			cq_num, frame_num, cq_addr_index);
> +	} else {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +	}
> +
> +	isp_dev->sof_count += 1;
> +
> +	return cq_num;
> +}
> +
> +static int irq_handle_notify_event(struct isp_device *isp_dev,
> +				   unsigned int irqstatus,
> +				   unsigned int dmastatus)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	unsigned long flags;
> +	int i;
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> +		(irqstatus & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> +		(dmastatus & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +
> +	dev_dbg(isp_dev->dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irqstatus & INT_ST_MASK_CAM),
> +		(dmastatus & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +
> +	return 0;
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cardinalnum, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev->streaming)
> +		return IRQ_HANDLED;
> +
> +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (irqstatus == 0)
> +		return IRQ_HANDLED;
> +
> +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> +
> +	/* sof , done order check . */
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> +		dev_warn(dev,
> +			 "isp sof_don block, sof_cnt:%d\n",
> +			 isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue ready frame */
> +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +		isp_dev->current_frame = hw_frame_num;
> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irqstatus & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +
> +		if ((irqstatus & INT_ST_MASK_CAM) ||
> +		    (dmastatus & DMA_ST_MASK_CAM))
> +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> +	}
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	if (irqstatus & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	if (irqstatus & SW_PASS1_DON_ST) {
> +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> +
> +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible
> +				(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	/* check ISP error status */
> +	if (errstatus) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x/0x%x\n",
> +			irqstatus, warnstatus, errstatus);
> +
> +		/* show DMA errors in detail */
> +		if (errstatus & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,
> +			'A' + cardinalnum,
> +			isp_dev->sof_count,
> +			irqstatus,
> +			dmastatus,
> +			hw_frame_num,
> +			cq_num);
> +	return IRQ_HANDLED;
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> +				      p1_dev->isp_clk.clk_list);
> +	if (ret < 0)
> +		goto clk_err;
> +	return 0;
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> +				   p1_dev->isp_clk.clk_list);
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int ret;
> +	unsigned int i;
> +
> +	/* Allocate context */
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;
> +
> +	p1_dev->isp_devs =
> +		devm_kzalloc(dev,
> +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	p1_dev->cam_dev =
> +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> +	if (!p1_dev->isp_devs)
> +		return -ENOMEM;
> +
> +	/* iomap registers */
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;
> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> +			 i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)
> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* reg & interrupts index is shifted with 1  */
> +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> +			if (isp_dev->irq > 0) {
> +				ret = devm_request_irq(dev, isp_dev->irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),
> +						       (void *)isp_dev);
> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						isp_dev->irq);
> +					return ret;
> +				}
> +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> +					 isp_dev->irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}
> +
> +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_clk.clk_list =
> +		devm_kcalloc(dev,
> +			     p1_dev->isp_clk.num_clks,
> +			     sizeof(*p1_dev->isp_clk.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_clk.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_clk.num_clks,
> +				p1_dev->isp_clk.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory\n");
> +		return ret;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int usercount, module;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* disable VF */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, The TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}
> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct isp_device *isp_dev;
> +	unsigned int reg_val;
> +	int module, usercount;
> +
> +	module = p1_dev->isp_ctx.isp_hw_module;
> +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> +
> +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> +
> +	/* If no user count, no further action */
> +	if (!usercount)
> +		return 0;
> +
> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev->streaming) {
> +		isp_dev = &p1_dev->isp_devs[module];
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}
> +	return 0;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> +		p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret < 0) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		mutex_init(&isp_ctx->composer_tx_lock);
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +	return 0;
> +}
> +
> +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> +
> +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)

I think s/enable/enabled would be a clearer name for both the function
and local variable.

> +{
> +	unsigned int enable_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enable_dma_ports = 0;
> +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> +		if (cam_dev->mem2mem2_nodes[i].enabled)
> +			enable_dma_ports |=
> +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> +	}
> +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> +		__func__, enable_dma_ports);
> +
> +	return enable_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return raw_pxl_id_b;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return raw_pxl_id_gb;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return raw_pxl_id_gr;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return raw_pxl_id_r;
> +	default:
> +		return raw_pxl_id_b;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return img_fmt_bayer8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return img_fmt_bayer10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return img_fmt_bayer12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return img_fmt_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return img_fmt_bayer8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return img_fmt_fg_bayer8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return img_fmt_bayer10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return img_fmt_fg_bayer10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return img_fmt_bayer12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return img_fmt_fg_bayer12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return img_fmt_bayer14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return img_fmt_fg_bayer14;
> +	default:
> +		return img_fmt_unknown;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	case V4L2_PIX_FMT_MTISP_U8:
> +	case V4L2_PIX_FMT_MTISP_U10:
> +	case V4L2_PIX_FMT_MTISP_U12:
> +	case V4L2_PIX_FMT_MTISP_U14:
> +		return 16;
> +	default:
> +		return 10;
> +	}
> +}
> +
> +static void composer_deinit_done_cb(void *data)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +
> +	disable_sys_clock(p1_dev);
> +	/* Notify PM */
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_open(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> +	int ret;
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> +
> +	if (usercount == 1) {
> +		ret = isp_setup_scp_rproc(p1_dev);
> +		if (ret)
> +			goto scp_err;
> +
> +		/* ISP HW INIT */
> +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> +		/* Use pure RAW as default HW path */
> +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +		/* Check enabled DMAs which is configured by media setup */
> +		isp_ctx->enable_dma_ports =
> +			get_enable_dma_ports(p1_dev->cam_dev);
> +
> +		if (!isp_ctx->enable_dma_ports) {
> +			dev_dbg(dev, "No DMAs are enabled\n");
> +			ret = -EINVAL;
> +			goto scp_err;
> +		}
> +
> +		pm_runtime_get_sync(dev);
> +
> +		ret = isp_init_context(p1_dev);
> +		if (ret)
> +			goto ctx_err;
> +		ret = isp_composer_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +		ret = isp_composer_hw_init(isp_ctx);
> +		if (ret)
> +			goto composer_err;
> +
> +		isp_composer_meta_config(&p1_dev->isp_ctx,
> +					 isp_ctx->enable_dma_ports);
> +	}
> +
> +	return 0;
> +composer_err:
> +	isp_uninit_context(p1_dev);
> +ctx_err:
> +	pm_runtime_put_sync(dev);
> +scp_err:
> +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> +	return ret;
> +}
> +
> +int mtk_isp_release(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> +		isp_uninit_context(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_format;
> +	unsigned int sd_width, sd_height;
> +	unsigned int enable_dma_ports, idx;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enable_dma_ports = isp_ctx->enable_dma_ports;
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> +
> +	/* sensor config */
> +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor,
> +			       pad, get_fmt, NULL, &sd_format);
> +
> +	if (ret) {
> +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	dev_dbg(dev,
> +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_format.format.width, sd_format.format.height,
> +		sd_format.format.code, sd_format.format.field,
> +		sd_format.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> +	/* support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;
> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_format.format.code);
> +	config_param.cfg_in_param.img_fmt =
> +		get_sensor_fmt(sd_format.format.code);
> +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> +	sd_width = sd_format.format.width;
> +	sd_height = sd_format.format.height;
> +
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;

The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
mem2mem2_nodes directly here and below.

> +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> +		struct v4l2_format *imgo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_param.cfg_main_param.bypass = 0;
> +
> +		config_param.cfg_main_param.output.img_fmt =
> +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.pixel_byte =
> +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_main_param.output.size.w =
> +			imgo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_main_param.output.size.h =
> +			imgo_fmt->fmt.pix_mp.height;
> +
> +		config_param.cfg_main_param.output.size.stride =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_main_param.output.size.xsize =
> +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_main_param.output.crop.left = 0x0;
> +		config_param.cfg_main_param.output.crop.top = 0x0;
> +
> +		config_param.cfg_main_param.output.crop.width = sd_width;
> +		config_param.cfg_main_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> +			  "img out:%d:%d in:%d:%d",
> +			  imgo_fmt->fmt.pix_mp.width,
> +			  imgo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev,
> +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> +			config_param.cfg_main_param.output.pixel_byte,
> +			config_param.cfg_main_param.output.img_fmt,
> +			config_param.cfg_main_param.pure_raw);
> +		dev_dbg(dev,
> +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_main_param.output.size.w,
> +			config_param.cfg_main_param.output.size.h,
> +			config_param.cfg_main_param.output.size.stride,
> +			config_param.cfg_main_param.output.size.xsize,
> +			config_param.cfg_main_param.output.crop.width,
> +			config_param.cfg_main_param.output.crop.height);
> +	} else {
> +		config_param.cfg_main_param.bypass = 1;
> +	}
> +
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> +		struct v4l2_format *rrzo_fmt =
> +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> +
> +		config_param.cfg_resize_param.bypass = 0;
> +		config_param.cfg_resize_param.output.img_fmt =
> +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.pixel_byte =
> +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> +		config_param.cfg_resize_param.output.size.w =
> +			rrzo_fmt->fmt.pix_mp.width;
> +		config_param.cfg_resize_param.output.size.h =
> +			rrzo_fmt->fmt.pix_mp.height;
> +		config_param.cfg_resize_param.output.size.stride =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +		config_param.cfg_resize_param.output.size.xsize =
> +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +		config_param.cfg_resize_param.output.crop.left = 0x0;
> +		config_param.cfg_resize_param.output.crop.top = 0x0;
> +		config_param.cfg_resize_param.output.crop.width = sd_width;
> +		config_param.cfg_resize_param.output.crop.height = sd_height;
> +
> +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> +			  "rrz out:%d:%d in:%d:%d",
> +			  rrzo_fmt->fmt.pix_mp.width,
> +			  rrzo_fmt->fmt.pix_mp.height,
> +			  sd_width,
> +			  sd_height);
> +
> +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> +			config_param.cfg_resize_param.output.pixel_byte,
> +			config_param.cfg_resize_param.output.img_fmt);
> +		dev_dbg(dev,
> +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> +			config_param.cfg_resize_param.output.size.w,
> +			config_param.cfg_resize_param.output.size.h,
> +			config_param.cfg_resize_param.output.size.stride,
> +			config_param.cfg_resize_param.output.size.xsize,
> +			config_param.cfg_resize_param.output.crop.width,
> +			config_param.cfg_resize_param.output.crop.height);
> +	} else {
> +		config_param.cfg_resize_param.bypass = 1;
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> +
> +	isp_composer_hw_config(isp_ctx, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +	return 0;
> +}
> +
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_port,
> +		    struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct mtk_cam_dev_buffer **bundle_buffers;
> +	unsigned int i, idx;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> +	memset(framejob, 0, sizeof(*framejob));
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	bundle_buffers = &frameparamsbase->buffers[0];
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> +	framejob->request_fd = frameparamsbase->request_fd;
> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	idx = MTK_CAM_P1_META_IN_0;
> +	if (bundle_buffers[idx]) {
> +		frameparams.tuning_addr.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.tuning_addr.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Image output */
> +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[0].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "main stream address iova:0x%x\n",
> +			frameparams.img_dma_buffers[0].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Resize output */
> +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> +	if (bundle_buffers[idx]) {
> +		frameparams.img_dma_buffers[1].buffer.iova =
> +			bundle_buffers[idx]->daddr;
> +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> +			bundle_buffers[idx]->scp_addr;
> +		dev_dbg(dev, "packed out address iova:0x%x\n",
> +			frameparams.img_dma_buffers[1].buffer.iova);
> +		list_add_tail(&bundle_buffers[idx]->list,
> +			      &framejob->list_buf);
> +	}
> +
> +	/* Meta output DMAs */
> +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> +		idx = MTK_CAM_P1_META_OUT_0 + i;
> +		if (bundle_buffers[idx]) {
> +			frameparams.meta_addrs[i].iova =
> +			  bundle_buffers[idx]->daddr;
> +			frameparams.meta_addrs[i].scp_addr =
> +			  bundle_buffers[idx]->scp_addr;
> +			list_add_tail(&bundle_buffers[idx]->list,
> +				      &framejob->list_buf);
> +		} else {
> +			frameparams.meta_addrs[i].iova = 0;
> +			frameparams.meta_addrs[i].scp_addr = 0;
> +		}
> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		frameparamsbase->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6cf8bb4ba93a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,300 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + * Author: Ryan Yu <ryan.yu@mediatek.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-dev.h"
> +#include "mtk_cam-scp.h"
> +
> +#define CAM_A_MAX_WIDTH		3328U
> +#define CAM_A_MAX_HEIGHT	2496U
> +#define CAM_B_MAX_WIDTH		5376U
> +#define CAM_B_MAX_HEIGHT	4032U
> +
> +#define CAM_MIN_WIDTH		80U
> +#define CAM_MIN_HEIGHT		60U
> +
> +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_PDO		BIT(6)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_RSSO		BIT(9)
> +#define R_PSO		BIT(10)
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_DATA_BUF_SIZE		4
> +#define COMPOSRE_EVENT_BUF_SIZE		4
> +
> +#define CQ_ADDRESS_OFFSET		0x640
> +#define CQ_BUFFER_COUNT			3

Please align macro values using tabs.

> +
> +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +enum {
> +	img_fmt_unknown		= 0x0000,
> +	img_fmt_raw_start	= 0x2200,
> +	img_fmt_bayer8		= img_fmt_raw_start,
> +	img_fmt_bayer10,
> +	img_fmt_bayer12,
> +	img_fmt_bayer14,
> +	img_fmt_fg_bayer8,
> +	img_fmt_fg_bayer10,
> +	img_fmt_fg_bayer12,
> +	img_fmt_fg_bayer14,
> +};
> +
> +enum {
> +	raw_pxl_id_b   = 0,
> +	raw_pxl_id_gb,
> +	raw_pxl_id_gr,
> +	raw_pxl_id_r
> +};
> +
> +enum {
> +	default_pixel_mode = 0,
> +	one_pixel_mode,
> +	two_pixel_mode,
> +	four_pixel_mode,
> +	pixel_mode_num,
> +};
> +
> +enum mtk_isp_scp_ipi_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};
> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_ipi_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};
> +
> +struct isp_clk_struct {
> +	int num_clks;
> +	struct clk_bulk_data *clk_list;
> +};
> +
> +struct isp_device {
> +	struct device *dev;
> +	void __iomem *regs;
> +	int irq;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	struct mutex composer_tx_lock; /* isp composer work protection */
> +
> +	struct isp_thread composer_rx_thread;
> +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> +	atomic_t composer_evts_start;
> +	atomic_t composer_evts_end;
> +	spinlock_t composer_evts_lock; /* SCP events protection */
> +	/* increase after ipi */
> +	atomic_t ipi_occupied;
> +	/* increase after frame enqueue */
> +	atomic_t composing_frame;
> +	/* current composed frame id */
> +	atomic_t composed_frame_id;
> +
> +	struct isp_queue p1_enqueue_list;
> +
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> +
> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	struct sg_table sgtable;
> +
> +	/* increase after open, decrease when close */
> +	atomic_t isp_user_cnt;
> +	/* frame sequence number, increase per en-queue*/
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;
> +	unsigned int enable_dma_ports;
> +
> +	void (*composer_deinit_donecb)(void *isp_ctx);
> +
> +	struct list_head list;
> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +
> +	/* for SCP driver  */
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct isp_clk_struct isp_clk;

What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
separate structs? They are only ever used in isp_p1_device.

> +	struct mtk_cam_dev *cam_dev;
> +	struct isp_device *isp_devs;

Similarly, why are these allocated at runtime rather then just members
of the struct?

In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
afterwards the struct mtk_cam_dev and struct isp_devices are. There will
only ever be ISP_DEV_NODE_NUM isp_devices.

Could this be changed to:
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
?

> +};
> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> +}
> +
> +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			      unsigned int dma);
> +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> +			    struct p1_config_param *config_param);
> +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> +			    void (*donecb)(void *data));
> +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> +			  void *data,
> +			  enum mtk_isp_scp_ipi_type type);

These functions are declared here, but implemented in mtk_cam-scp.c.
Can the funtion declarations be moved to mtk_cam-scp.h?

> +
> +/**
> + * mtk_isp_open - open isp driver and initialize related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_open(struct device *dev);
> +
> +/**
> + * mtk_isp_release - release isp driver and related resources.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_release(struct device *dev);
> +
> +/**
> + * mtk_isp_config - output image & meta data configuration.
> + *
> + * @dev:	isp device.
> + *
> + */
> +int mtk_isp_config(struct device *dev);
> +
> +/**
> + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> + *
> + * @dev:	isp device.
> + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> + *
> + */
> +int mtk_isp_req_enqueue(struct device *dev,
> +			struct mtk_cam_dev_start_param *frameparamsbase);
> +
> +/**
> + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> + * for non-per-frame DMA.
> + *
> + * @dev:	isp device.
> + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> + *
> + */
> +int mtk_isp_enqueue(struct device *dev,
> +		    unsigned int dma_idx,
> +		    struct mtk_cam_dev_buffer *buffer);
> +#endif /*__CAMERA_ISP_H*/
> -- 
> 2.18.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 09/11] media: platform: Add Mediatek ISP P1 device driver
  2019-05-24 21:19     ` [RFC,V2,09/11] " Drew Davenport
  (?)
@ 2019-05-27 13:07       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-27 13:07 UTC (permalink / raw)
  To: Drew Davenport
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, frankie.chiu, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Drew:


On Fri, 2019-05-24 at 15:19 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> 
> A few comments inline.
> 

Appreciate your feedback on this patch set firstly.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
> >  3 files changed, 1655 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..342f0e0e9837
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,149 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT	BIT(0)
> > +#define CMOS_EN_BIT	BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST	BIT(0)
> > +#define HW_PASS1_DON_ST	BIT(11)
> > +#define SOF_INT_ST	BIT(12)
> > +#define SW_PASS1_DON_ST	BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST	BIT(4)
> > +#define TG_GBERR_ST	BIT(5)
> > +#define CQ_CODE_ERR_ST	BIT(6)
> > +#define CQ_APB_ERR_ST	BIT(7)
> > +#define CQ_VS_ERR_ST	BIT(8)
> > +#define AMX_ERR_ST	BIT(15)
> > +#define RMX_ERR_ST	BIT(16)
> > +#define BMX_ERR_ST	BIT(17)
> > +#define RRZO_ERR_ST	BIT(18)
> > +#define AFO_ERR_ST	BIT(19)
> > +#define IMGO_ERR_ST	BIT(20)
> > +#define AAO_ERR_ST	BIT(21)
> > +#define PSO_ERR_ST	BIT(22)
> > +#define LCSO_ERR_ST	BIT(23)
> > +#define BNR_ERR_ST	BIT(24)
> > +#define LSCI_ERR_ST	BIT(25)
> > +#define DMA_ERR_ST	BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST	BIT(4)
> > +#define AFO_DONE_ST	BIT(5)
> > +#define AAO_DONE_ST	BIT(7)
> > +#define PSO_DONE_ST	BIT(14)
> 
> Please align the values using tabs here and elsewhere.
> 

Ok, we will revise this coding style issue in next patch.

> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM	( \
> > +			VS_INT_ST |\
> > +			SOF_INT_ST |\
> > +			HW_PASS1_DON_ST |\
> > +			SW_PASS1_DON_ST)
> > +
> > +/* IRQ Warning Mask */
> > +#define INT_ST_MASK_CAM_WARN	(\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				PSO_ERR_ST | \
> > +				LCSO_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR	(\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				CQ_CODE_ERR_ST |\
> > +				CQ_APB_ERR_ST |\
> > +				CQ_VS_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				RMX_ERR_ST |\
> > +				BMX_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM	(\
> > +				SOF_INT_ST |\
> > +				SW_PASS1_DON_ST |\
> > +				VS_INT_ST |\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM	(\
> > +			AFO_DONE_ST |\
> > +			AAO_DONE_ST |\
> > +			PSO_DONE_ST |\
> > +			FLKO_DONE_ST)
> > +
> > +/* Status check */
> > +#define REG_CTL_EN		0x0004
> > +#define REG_CTL_DMA_EN		0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2		0x0018
> > +#define REG_CTL_RAW_INT_EN	0x0020
> > +#define REG_CTL_RAW_INT_STAT	0x0024
> > +#define REG_CTL_RAW_INT2_STAT	0x0034
> > +#define REG_CTL_RAW_INT3_STAT	0x00c4
> > +#define REG_CTL_TWIN_STAT	0x0050
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_SEN_GRAB_PIX	0x0238
> > +#define REG_TG_SEN_GRAB_LIN	0x023c
> > +#define REG_TG_VF_CON		0x0234
> > +#define REG_TG_SUB_PERIOD	0x02a4
> > +
> > +#define REG_IMGO_BASE_ADDR	0x1020
> > +#define REG_RRZO_BASE_ADDR	0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT	0x1360
> > +#define REG_RRZO_ERR_STAT	0x1364
> > +#define REG_AAO_ERR_STAT	0x1368
> > +#define REG_AFO_ERR_STAT	0x136c
> > +#define REG_LCSO_ERR_STAT	0x1370
> > +#define REG_UFEO_ERR_STAT	0x1374
> > +#define REG_PDO_ERR_STAT	0x1378
> > +#define REG_BPCI_ERR_STAT	0x137c
> > +#define REG_LSCI_ERR_STAT	0x1384
> > +#define REG_PDI_ERR_STAT	0x138c
> > +#define REG_LMVO_ERR_STAT	0x1390
> > +#define REG_FLKO_ERR_STAT	0x1394
> > +#define REG_PSO_ERR_STAT	0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR	0x0198
> > +#define REG_HW_FRAME_NUM	0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX	0x14dc
> > +#define REG_META1_VB2_INDEX	0x151c
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..fc874ec8f7f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1206 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +/* list of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> > +};
> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> > +			   unsigned int request_fd,
> > +			   unsigned int frame_seq_no,
> > +			   struct list_head *list_buf,
> > +			   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev_finish_param fram_param;
> > +
> > +	fram_param.list_buf = list_buf;
> > +	fram_param.request_fd = request_fd;
> > +	fram_param.frame_seq_no = frame_seq_no;
> > +	fram_param.state = state;
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> > +		fram_param.request_fd,
> > +		fram_param.frame_seq_no);
> > +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> > +	struct vb2_buffer *vb;
> > +	struct vb2_v4l2_buffer *vbb;
> > +
> > +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> > +		return;
> > +
> > +	mutex_lock(vb2_queue->lock);
> > +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> > +		vbb = to_vb2_v4l2_buffer(vb);
> > +		if (vbb->request_fd < 0 &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vbb->vb2_buf.timestamp = ktime_get_ns();
> > +			vbb->sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +		}
> > +	}
> > +	mutex_unlock(vb2_queue->lock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		if (ret == ERESTARTSYS) {
> > +			dev_err(dev, "interrupted by a signal!\n");
> > +			continue;
> > +		}
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & VS_INT_ST) {
> > +			/* Notify specific HW events to user space */
> > +			mtk_cam_dev_event_frame_sync(cam_dev,
> > +						     event_data.frame_seq_no);
> > +			dev_dbg(dev,
> > +				"event IRQ:0x%x DMA:0x%x is sent\n",
> > +				event_data.irq_status_mask,
> > +				event_data.dma_status_mask);
> > +		}
> > +
> > +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int cq_addr_index;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	if (cq_num > frame_num) {
> > +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> > +
> > +		writel(base_addr +
> > +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> > +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> > +			cq_num, frame_num, cq_addr_index);
> > +	} else {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +	}
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	return cq_num;
> > +}
> > +
> > +static int irq_handle_notify_event(struct isp_device *isp_dev,
> > +				   unsigned int irqstatus,
> > +				   unsigned int dmastatus)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> > +		(irqstatus & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> > +		(dmastatus & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +
> > +	dev_dbg(isp_dev->dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irqstatus & INT_ST_MASK_CAM),
> > +		(dmastatus & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +
> > +	return 0;
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cardinalnum, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev->streaming)
> > +		return IRQ_HANDLED;
> > +
> > +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (irqstatus == 0)
> > +		return IRQ_HANDLED;
> > +
> > +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> > +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> > +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> > +
> > +	/* sof , done order check . */
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> > +		dev_warn(dev,
> > +			 "isp sof_don block, sof_cnt:%d\n",
> > +			 isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue ready frame */
> > +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +		isp_dev->current_frame = hw_frame_num;
> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irqstatus & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +
> > +		if ((irqstatus & INT_ST_MASK_CAM) ||
> > +		    (dmastatus & DMA_ST_MASK_CAM))
> > +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +	}
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	if (irqstatus & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	if (irqstatus & SW_PASS1_DON_ST) {
> > +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> > +
> > +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible
> > +				(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	/* check ISP error status */
> > +	if (errstatus) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x/0x%x\n",
> > +			irqstatus, warnstatus, errstatus);
> > +
> > +		/* show DMA errors in detail */
> > +		if (errstatus & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> > +			'A' + cardinalnum,
> > +			isp_dev->sof_count,
> > +			irqstatus,
> > +			dmastatus,
> > +			hw_frame_num,
> > +			cq_num);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> > +				      p1_dev->isp_clk.clk_list);
> > +	if (ret < 0)
> > +		goto clk_err;
> > +	return 0;
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	/* Allocate context */
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> > +
> > +	p1_dev->isp_devs =
> > +		devm_kzalloc(dev,
> > +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->cam_dev =
> > +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	/* iomap registers */
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> > +			 i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* reg & interrupts index is shifted with 1  */
> > +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> > +			if (isp_dev->irq > 0) {
> > +				ret = devm_request_irq(dev, isp_dev->irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> > +						       (void *)isp_dev);
> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						isp_dev->irq);
> > +					return ret;
> > +				}
> > +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> > +					 isp_dev->irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> > +
> > +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_clk.clk_list =
> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_clk.num_clks,
> > +			     sizeof(*p1_dev->isp_clk.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_clk.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> > +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_clk.num_clks,
> > +				p1_dev->isp_clk.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory\n");
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int usercount, module;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* disable VF */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, The TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int module, usercount;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev->streaming) {
> > +		isp_dev = &p1_dev->isp_devs[module];
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> > +		p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret < 0) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		mutex_init(&isp_ctx->composer_tx_lock);
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +	return 0;
> > +}
> > +
> > +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> > +
> > +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
> 
> I think s/enable/enabled would be a clearer name for both the function
> and local variable.
> 

Ok, we will revised this "enabled" working in next patch.

> > +{
> > +	unsigned int enable_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enable_dma_ports = 0;
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		if (cam_dev->mem2mem2_nodes[i].enabled)
> > +			enable_dma_ports |=
> > +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> > +	}
> > +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> > +		__func__, enable_dma_ports);
> > +
> > +	return enable_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return raw_pxl_id_b;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return raw_pxl_id_gb;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return raw_pxl_id_gr;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return raw_pxl_id_r;
> > +	default:
> > +		return raw_pxl_id_b;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return img_fmt_bayer8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return img_fmt_bayer10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return img_fmt_bayer12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return img_fmt_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return img_fmt_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return img_fmt_fg_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return img_fmt_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return img_fmt_fg_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return img_fmt_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return img_fmt_fg_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return img_fmt_bayer14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return img_fmt_fg_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 10;
> > +	}
> > +}
> > +
> > +static void composer_deinit_done_cb(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +
> > +	disable_sys_clock(p1_dev);
> > +	/* Notify PM */
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_open(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> > +
> > +	if (usercount == 1) {
> > +		ret = isp_setup_scp_rproc(p1_dev);
> > +		if (ret)
> > +			goto scp_err;
> > +
> > +		/* ISP HW INIT */
> > +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> > +		/* Use pure RAW as default HW path */
> > +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +		/* Check enabled DMAs which is configured by media setup */
> > +		isp_ctx->enable_dma_ports =
> > +			get_enable_dma_ports(p1_dev->cam_dev);
> > +
> > +		if (!isp_ctx->enable_dma_ports) {
> > +			dev_dbg(dev, "No DMAs are enabled\n");
> > +			ret = -EINVAL;
> > +			goto scp_err;
> > +		}
> > +
> > +		pm_runtime_get_sync(dev);
> > +
> > +		ret = isp_init_context(p1_dev);
> > +		if (ret)
> > +			goto ctx_err;
> > +		ret = isp_composer_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +		ret = isp_composer_hw_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +
> > +		isp_composer_meta_config(&p1_dev->isp_ctx,
> > +					 isp_ctx->enable_dma_ports);
> > +	}
> > +
> > +	return 0;
> > +composer_err:
> > +	isp_uninit_context(p1_dev);
> > +ctx_err:
> > +	pm_runtime_put_sync(dev);
> > +scp_err:
> > +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_release(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> > +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> > +		isp_uninit_context(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> > +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_format;
> > +	unsigned int sd_width, sd_height;
> > +	unsigned int enable_dma_ports, idx;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enable_dma_ports = isp_ctx->enable_dma_ports;
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> > +
> > +	/* sensor config */
> > +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor,
> > +			       pad, get_fmt, NULL, &sd_format);
> > +
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_format.format.width, sd_format.format.height,
> > +		sd_format.format.code, sd_format.format.field,
> > +		sd_format.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> > +	/* support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_format.format.code);
> > +	config_param.cfg_in_param.img_fmt =
> > +		get_sensor_fmt(sd_format.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> > +	sd_width = sd_format.format.width;
> > +	sd_height = sd_format.format.height;
> > +
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> 
> The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
> mem2mem2_nodes directly here and below.
> 

Ok, we will remove idx variable and use const value to index.

> > +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> > +		struct v4l2_format *imgo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_param.cfg_main_param.bypass = 0;
> > +
> > +		config_param.cfg_main_param.output.img_fmt =
> > +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.pixel_byte =
> > +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.size.w =
> > +			imgo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_main_param.output.size.h =
> > +			imgo_fmt->fmt.pix_mp.height;
> > +
> > +		config_param.cfg_main_param.output.size.stride =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_main_param.output.size.xsize =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_main_param.output.crop.left = 0x0;
> > +		config_param.cfg_main_param.output.crop.top = 0x0;
> > +
> > +		config_param.cfg_main_param.output.crop.width = sd_width;
> > +		config_param.cfg_main_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "img out:%d:%d in:%d:%d",
> > +			  imgo_fmt->fmt.pix_mp.width,
> > +			  imgo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev,
> > +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> > +			config_param.cfg_main_param.output.pixel_byte,
> > +			config_param.cfg_main_param.output.img_fmt,
> > +			config_param.cfg_main_param.pure_raw);
> > +		dev_dbg(dev,
> > +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_main_param.output.size.w,
> > +			config_param.cfg_main_param.output.size.h,
> > +			config_param.cfg_main_param.output.size.stride,
> > +			config_param.cfg_main_param.output.size.xsize,
> > +			config_param.cfg_main_param.output.crop.width,
> > +			config_param.cfg_main_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_main_param.bypass = 1;
> > +	}
> > +
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> > +		struct v4l2_format *rrzo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_param.cfg_resize_param.output.img_fmt =
> > +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.pixel_byte =
> > +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.size.w =
> > +			rrzo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_resize_param.output.size.h =
> > +			rrzo_fmt->fmt.pix_mp.height;
> > +		config_param.cfg_resize_param.output.size.stride =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_resize_param.output.size.xsize =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_resize_param.output.crop.left = 0x0;
> > +		config_param.cfg_resize_param.output.crop.top = 0x0;
> > +		config_param.cfg_resize_param.output.crop.width = sd_width;
> > +		config_param.cfg_resize_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "rrz out:%d:%d in:%d:%d",
> > +			  rrzo_fmt->fmt.pix_mp.width,
> > +			  rrzo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> > +			config_param.cfg_resize_param.output.pixel_byte,
> > +			config_param.cfg_resize_param.output.img_fmt);
> > +		dev_dbg(dev,
> > +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_resize_param.output.size.w,
> > +			config_param.cfg_resize_param.output.size.h,
> > +			config_param.cfg_resize_param.output.size.stride,
> > +			config_param.cfg_resize_param.output.size.xsize,
> > +			config_param.cfg_resize_param.output.crop.width,
> > +			config_param.cfg_resize_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = 1;
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> > +
> > +	isp_composer_hw_config(isp_ctx, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_port,
> > +		    struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct mtk_cam_dev_buffer **bundle_buffers;
> > +	unsigned int i, idx;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > +	memset(framejob, 0, sizeof(*framejob));
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	bundle_buffers = &frameparamsbase->buffers[0];
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> > +	framejob->request_fd = frameparamsbase->request_fd;
> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	idx = MTK_CAM_P1_META_IN_0;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.tuning_addr.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.tuning_addr.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Image output */
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[0].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "main stream address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[0].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Resize output */
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[1].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "packed out address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[1].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Meta output DMAs */
> > +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> > +		idx = MTK_CAM_P1_META_OUT_0 + i;
> > +		if (bundle_buffers[idx]) {
> > +			frameparams.meta_addrs[i].iova =
> > +			  bundle_buffers[idx]->daddr;
> > +			frameparams.meta_addrs[i].scp_addr =
> > +			  bundle_buffers[idx]->scp_addr;
> > +			list_add_tail(&bundle_buffers[idx]->list,
> > +				      &framejob->list_buf);
> > +		} else {
> > +			frameparams.meta_addrs[i].iova = 0;
> > +			frameparams.meta_addrs[i].scp_addr = 0;
> > +		}
> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		frameparamsbase->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> > +MODULE_LICENSE("GPL");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..6cf8bb4ba93a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,300 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-scp.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328U
> > +#define CAM_A_MAX_HEIGHT	2496U
> > +#define CAM_B_MAX_WIDTH		5376U
> > +#define CAM_B_MAX_HEIGHT	4032U
> > +
> > +#define CAM_MIN_WIDTH		80U
> > +#define CAM_MIN_HEIGHT		60U
> > +
> > +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_PDO		BIT(6)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_RSSO		BIT(9)
> > +#define R_PSO		BIT(10)
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define COMPOSRE_EVENT_BUF_SIZE		4
> > +
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +#define CQ_BUFFER_COUNT			3
> 
> Please align macro values using tabs.
> 
> > +
> > +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +enum {
> > +	img_fmt_unknown		= 0x0000,
> > +	img_fmt_raw_start	= 0x2200,
> > +	img_fmt_bayer8		= img_fmt_raw_start,
> > +	img_fmt_bayer10,
> > +	img_fmt_bayer12,
> > +	img_fmt_bayer14,
> > +	img_fmt_fg_bayer8,
> > +	img_fmt_fg_bayer10,
> > +	img_fmt_fg_bayer12,
> > +	img_fmt_fg_bayer14,
> > +};
> > +
> > +enum {
> > +	raw_pxl_id_b   = 0,
> > +	raw_pxl_id_gb,
> > +	raw_pxl_id_gr,
> > +	raw_pxl_id_r
> > +};
> > +
> > +enum {
> > +	default_pixel_mode = 0,
> > +	one_pixel_mode,
> > +	two_pixel_mode,
> > +	four_pixel_mode,
> > +	pixel_mode_num,
> > +};
> > +
> > +enum mtk_isp_scp_ipi_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_ipi_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> > +
> > +struct isp_clk_struct {
> > +	int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +};
> > +
> > +struct isp_device {
> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	int irq;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	struct mutex composer_tx_lock; /* isp composer work protection */
> > +
> > +	struct isp_thread composer_rx_thread;
> > +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> > +	atomic_t composer_evts_start;
> > +	atomic_t composer_evts_end;
> > +	spinlock_t composer_evts_lock; /* SCP events protection */
> > +	/* increase after ipi */
> > +	atomic_t ipi_occupied;
> > +	/* increase after frame enqueue */
> > +	atomic_t composing_frame;
> > +	/* current composed frame id */
> > +	atomic_t composed_frame_id;
> > +
> > +	struct isp_queue p1_enqueue_list;
> > +
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> > +
> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	struct sg_table sgtable;
> > +
> > +	/* increase after open, decrease when close */
> > +	atomic_t isp_user_cnt;
> > +	/* frame sequence number, increase per en-queue*/
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> > +	unsigned int enable_dma_ports;
> > +
> > +	void (*composer_deinit_donecb)(void *isp_ctx);
> > +
> > +	struct list_head list;
> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +
> > +	/* for SCP driver  */
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct isp_clk_struct isp_clk;
> 
> What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
> separate structs? They are only ever used in isp_p1_device.
> 

Ok, we will merge isp_clk_struct into mtk_isp_p1_ctx in next patch.

> > +	struct mtk_cam_dev *cam_dev;
> > +	struct isp_device *isp_devs;
> 
> Similarly, why are these allocated at runtime rather then just members
> of the struct?
> 

Ok, we will revise new isp_p1_device structure as below list:

struct isp_p1_device {
	struct platform_device *pdev;
	struct platform_device *scp_pdev;
	struct rproc *rproc_handle;
	struct mtk_isp_p1_ctx isp_ctx;
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
};


> In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
> afterwards the struct mtk_cam_dev and struct isp_devices are. There will
> only ever be ISP_DEV_NODE_NUM isp_devices.
> 
> Could this be changed to:
> 	struct mtk_cam_dev cam_dev;
> 	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> ?
> 

Yes, thanks for your suggestion as above.

> > +};
> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> > +}
> > +
> > +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			      unsigned int dma);
> > +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    struct p1_config_param *config_param);
> > +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> > +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    void (*donecb)(void *data));
> > +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> > +			  void *data,
> > +			  enum mtk_isp_scp_ipi_type type);
> 
> These functions are declared here, but implemented in mtk_cam-scp.c.
> Can the funtion declarations be moved to mtk_cam-scp.h?
> 

Ok, we will revise these function declared in next patch.

Best regards,


Jungo 

> > +
> > +/**
> > + * mtk_isp_open - open isp driver and initialize related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_open(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_release - release isp driver and related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_release(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_config - output image & meta data configuration.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_config(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> > + *
> > + * @dev:	isp device.
> > + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> > + *
> > + */
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase);
> > +
> > +/**
> > + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> > + * for non-per-frame DMA.
> > + *
> > + * @dev:	isp device.
> > + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> > + *
> > + */
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_idx,
> > +		    struct mtk_cam_dev_buffer *buffer);
> > +#endif /*__CAMERA_ISP_H*/
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-27 13:07       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-27 13:07 UTC (permalink / raw)
  To: Drew Davenport
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi, Drew:


On Fri, 2019-05-24 at 15:19 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> 
> A few comments inline.
> 

Appreciate your feedback on this patch set firstly.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
> >  3 files changed, 1655 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..342f0e0e9837
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,149 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT	BIT(0)
> > +#define CMOS_EN_BIT	BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST	BIT(0)
> > +#define HW_PASS1_DON_ST	BIT(11)
> > +#define SOF_INT_ST	BIT(12)
> > +#define SW_PASS1_DON_ST	BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST	BIT(4)
> > +#define TG_GBERR_ST	BIT(5)
> > +#define CQ_CODE_ERR_ST	BIT(6)
> > +#define CQ_APB_ERR_ST	BIT(7)
> > +#define CQ_VS_ERR_ST	BIT(8)
> > +#define AMX_ERR_ST	BIT(15)
> > +#define RMX_ERR_ST	BIT(16)
> > +#define BMX_ERR_ST	BIT(17)
> > +#define RRZO_ERR_ST	BIT(18)
> > +#define AFO_ERR_ST	BIT(19)
> > +#define IMGO_ERR_ST	BIT(20)
> > +#define AAO_ERR_ST	BIT(21)
> > +#define PSO_ERR_ST	BIT(22)
> > +#define LCSO_ERR_ST	BIT(23)
> > +#define BNR_ERR_ST	BIT(24)
> > +#define LSCI_ERR_ST	BIT(25)
> > +#define DMA_ERR_ST	BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST	BIT(4)
> > +#define AFO_DONE_ST	BIT(5)
> > +#define AAO_DONE_ST	BIT(7)
> > +#define PSO_DONE_ST	BIT(14)
> 
> Please align the values using tabs here and elsewhere.
> 

Ok, we will revise this coding style issue in next patch.

> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM	( \
> > +			VS_INT_ST |\
> > +			SOF_INT_ST |\
> > +			HW_PASS1_DON_ST |\
> > +			SW_PASS1_DON_ST)
> > +
> > +/* IRQ Warning Mask */
> > +#define INT_ST_MASK_CAM_WARN	(\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				PSO_ERR_ST | \
> > +				LCSO_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR	(\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				CQ_CODE_ERR_ST |\
> > +				CQ_APB_ERR_ST |\
> > +				CQ_VS_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				RMX_ERR_ST |\
> > +				BMX_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM	(\
> > +				SOF_INT_ST |\
> > +				SW_PASS1_DON_ST |\
> > +				VS_INT_ST |\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM	(\
> > +			AFO_DONE_ST |\
> > +			AAO_DONE_ST |\
> > +			PSO_DONE_ST |\
> > +			FLKO_DONE_ST)
> > +
> > +/* Status check */
> > +#define REG_CTL_EN		0x0004
> > +#define REG_CTL_DMA_EN		0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2		0x0018
> > +#define REG_CTL_RAW_INT_EN	0x0020
> > +#define REG_CTL_RAW_INT_STAT	0x0024
> > +#define REG_CTL_RAW_INT2_STAT	0x0034
> > +#define REG_CTL_RAW_INT3_STAT	0x00c4
> > +#define REG_CTL_TWIN_STAT	0x0050
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_SEN_GRAB_PIX	0x0238
> > +#define REG_TG_SEN_GRAB_LIN	0x023c
> > +#define REG_TG_VF_CON		0x0234
> > +#define REG_TG_SUB_PERIOD	0x02a4
> > +
> > +#define REG_IMGO_BASE_ADDR	0x1020
> > +#define REG_RRZO_BASE_ADDR	0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT	0x1360
> > +#define REG_RRZO_ERR_STAT	0x1364
> > +#define REG_AAO_ERR_STAT	0x1368
> > +#define REG_AFO_ERR_STAT	0x136c
> > +#define REG_LCSO_ERR_STAT	0x1370
> > +#define REG_UFEO_ERR_STAT	0x1374
> > +#define REG_PDO_ERR_STAT	0x1378
> > +#define REG_BPCI_ERR_STAT	0x137c
> > +#define REG_LSCI_ERR_STAT	0x1384
> > +#define REG_PDI_ERR_STAT	0x138c
> > +#define REG_LMVO_ERR_STAT	0x1390
> > +#define REG_FLKO_ERR_STAT	0x1394
> > +#define REG_PSO_ERR_STAT	0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR	0x0198
> > +#define REG_HW_FRAME_NUM	0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX	0x14dc
> > +#define REG_META1_VB2_INDEX	0x151c
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..fc874ec8f7f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1206 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +/* list of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> > +};
> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> > +			   unsigned int request_fd,
> > +			   unsigned int frame_seq_no,
> > +			   struct list_head *list_buf,
> > +			   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev_finish_param fram_param;
> > +
> > +	fram_param.list_buf = list_buf;
> > +	fram_param.request_fd = request_fd;
> > +	fram_param.frame_seq_no = frame_seq_no;
> > +	fram_param.state = state;
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> > +		fram_param.request_fd,
> > +		fram_param.frame_seq_no);
> > +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> > +	struct vb2_buffer *vb;
> > +	struct vb2_v4l2_buffer *vbb;
> > +
> > +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> > +		return;
> > +
> > +	mutex_lock(vb2_queue->lock);
> > +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> > +		vbb = to_vb2_v4l2_buffer(vb);
> > +		if (vbb->request_fd < 0 &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vbb->vb2_buf.timestamp = ktime_get_ns();
> > +			vbb->sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +		}
> > +	}
> > +	mutex_unlock(vb2_queue->lock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		if (ret == ERESTARTSYS) {
> > +			dev_err(dev, "interrupted by a signal!\n");
> > +			continue;
> > +		}
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & VS_INT_ST) {
> > +			/* Notify specific HW events to user space */
> > +			mtk_cam_dev_event_frame_sync(cam_dev,
> > +						     event_data.frame_seq_no);
> > +			dev_dbg(dev,
> > +				"event IRQ:0x%x DMA:0x%x is sent\n",
> > +				event_data.irq_status_mask,
> > +				event_data.dma_status_mask);
> > +		}
> > +
> > +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int cq_addr_index;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	if (cq_num > frame_num) {
> > +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> > +
> > +		writel(base_addr +
> > +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> > +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> > +			cq_num, frame_num, cq_addr_index);
> > +	} else {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +	}
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	return cq_num;
> > +}
> > +
> > +static int irq_handle_notify_event(struct isp_device *isp_dev,
> > +				   unsigned int irqstatus,
> > +				   unsigned int dmastatus)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> > +		(irqstatus & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> > +		(dmastatus & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +
> > +	dev_dbg(isp_dev->dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irqstatus & INT_ST_MASK_CAM),
> > +		(dmastatus & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +
> > +	return 0;
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cardinalnum, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev->streaming)
> > +		return IRQ_HANDLED;
> > +
> > +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (irqstatus == 0)
> > +		return IRQ_HANDLED;
> > +
> > +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> > +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> > +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> > +
> > +	/* sof , done order check . */
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> > +		dev_warn(dev,
> > +			 "isp sof_don block, sof_cnt:%d\n",
> > +			 isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue ready frame */
> > +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +		isp_dev->current_frame = hw_frame_num;
> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irqstatus & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +
> > +		if ((irqstatus & INT_ST_MASK_CAM) ||
> > +		    (dmastatus & DMA_ST_MASK_CAM))
> > +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +	}
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	if (irqstatus & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	if (irqstatus & SW_PASS1_DON_ST) {
> > +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> > +
> > +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible
> > +				(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	/* check ISP error status */
> > +	if (errstatus) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x/0x%x\n",
> > +			irqstatus, warnstatus, errstatus);
> > +
> > +		/* show DMA errors in detail */
> > +		if (errstatus & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> > +			'A' + cardinalnum,
> > +			isp_dev->sof_count,
> > +			irqstatus,
> > +			dmastatus,
> > +			hw_frame_num,
> > +			cq_num);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> > +				      p1_dev->isp_clk.clk_list);
> > +	if (ret < 0)
> > +		goto clk_err;
> > +	return 0;
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	/* Allocate context */
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> > +
> > +	p1_dev->isp_devs =
> > +		devm_kzalloc(dev,
> > +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->cam_dev =
> > +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	/* iomap registers */
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> > +			 i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* reg & interrupts index is shifted with 1  */
> > +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> > +			if (isp_dev->irq > 0) {
> > +				ret = devm_request_irq(dev, isp_dev->irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> > +						       (void *)isp_dev);
> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						isp_dev->irq);
> > +					return ret;
> > +				}
> > +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> > +					 isp_dev->irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> > +
> > +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_clk.clk_list =
> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_clk.num_clks,
> > +			     sizeof(*p1_dev->isp_clk.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_clk.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> > +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_clk.num_clks,
> > +				p1_dev->isp_clk.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory\n");
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int usercount, module;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* disable VF */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, The TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int module, usercount;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev->streaming) {
> > +		isp_dev = &p1_dev->isp_devs[module];
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> > +		p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret < 0) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		mutex_init(&isp_ctx->composer_tx_lock);
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +	return 0;
> > +}
> > +
> > +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> > +
> > +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
> 
> I think s/enable/enabled would be a clearer name for both the function
> and local variable.
> 

Ok, we will revised this "enabled" working in next patch.

> > +{
> > +	unsigned int enable_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enable_dma_ports = 0;
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		if (cam_dev->mem2mem2_nodes[i].enabled)
> > +			enable_dma_ports |=
> > +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> > +	}
> > +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> > +		__func__, enable_dma_ports);
> > +
> > +	return enable_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return raw_pxl_id_b;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return raw_pxl_id_gb;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return raw_pxl_id_gr;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return raw_pxl_id_r;
> > +	default:
> > +		return raw_pxl_id_b;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return img_fmt_bayer8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return img_fmt_bayer10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return img_fmt_bayer12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return img_fmt_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return img_fmt_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return img_fmt_fg_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return img_fmt_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return img_fmt_fg_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return img_fmt_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return img_fmt_fg_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return img_fmt_bayer14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return img_fmt_fg_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 10;
> > +	}
> > +}
> > +
> > +static void composer_deinit_done_cb(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +
> > +	disable_sys_clock(p1_dev);
> > +	/* Notify PM */
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_open(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> > +
> > +	if (usercount == 1) {
> > +		ret = isp_setup_scp_rproc(p1_dev);
> > +		if (ret)
> > +			goto scp_err;
> > +
> > +		/* ISP HW INIT */
> > +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> > +		/* Use pure RAW as default HW path */
> > +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +		/* Check enabled DMAs which is configured by media setup */
> > +		isp_ctx->enable_dma_ports =
> > +			get_enable_dma_ports(p1_dev->cam_dev);
> > +
> > +		if (!isp_ctx->enable_dma_ports) {
> > +			dev_dbg(dev, "No DMAs are enabled\n");
> > +			ret = -EINVAL;
> > +			goto scp_err;
> > +		}
> > +
> > +		pm_runtime_get_sync(dev);
> > +
> > +		ret = isp_init_context(p1_dev);
> > +		if (ret)
> > +			goto ctx_err;
> > +		ret = isp_composer_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +		ret = isp_composer_hw_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +
> > +		isp_composer_meta_config(&p1_dev->isp_ctx,
> > +					 isp_ctx->enable_dma_ports);
> > +	}
> > +
> > +	return 0;
> > +composer_err:
> > +	isp_uninit_context(p1_dev);
> > +ctx_err:
> > +	pm_runtime_put_sync(dev);
> > +scp_err:
> > +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_release(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> > +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> > +		isp_uninit_context(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> > +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_format;
> > +	unsigned int sd_width, sd_height;
> > +	unsigned int enable_dma_ports, idx;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enable_dma_ports = isp_ctx->enable_dma_ports;
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> > +
> > +	/* sensor config */
> > +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor,
> > +			       pad, get_fmt, NULL, &sd_format);
> > +
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_format.format.width, sd_format.format.height,
> > +		sd_format.format.code, sd_format.format.field,
> > +		sd_format.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> > +	/* support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_format.format.code);
> > +	config_param.cfg_in_param.img_fmt =
> > +		get_sensor_fmt(sd_format.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> > +	sd_width = sd_format.format.width;
> > +	sd_height = sd_format.format.height;
> > +
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> 
> The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
> mem2mem2_nodes directly here and below.
> 

Ok, we will remove idx variable and use const value to index.

> > +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> > +		struct v4l2_format *imgo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_param.cfg_main_param.bypass = 0;
> > +
> > +		config_param.cfg_main_param.output.img_fmt =
> > +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.pixel_byte =
> > +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.size.w =
> > +			imgo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_main_param.output.size.h =
> > +			imgo_fmt->fmt.pix_mp.height;
> > +
> > +		config_param.cfg_main_param.output.size.stride =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_main_param.output.size.xsize =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_main_param.output.crop.left = 0x0;
> > +		config_param.cfg_main_param.output.crop.top = 0x0;
> > +
> > +		config_param.cfg_main_param.output.crop.width = sd_width;
> > +		config_param.cfg_main_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "img out:%d:%d in:%d:%d",
> > +			  imgo_fmt->fmt.pix_mp.width,
> > +			  imgo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev,
> > +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> > +			config_param.cfg_main_param.output.pixel_byte,
> > +			config_param.cfg_main_param.output.img_fmt,
> > +			config_param.cfg_main_param.pure_raw);
> > +		dev_dbg(dev,
> > +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_main_param.output.size.w,
> > +			config_param.cfg_main_param.output.size.h,
> > +			config_param.cfg_main_param.output.size.stride,
> > +			config_param.cfg_main_param.output.size.xsize,
> > +			config_param.cfg_main_param.output.crop.width,
> > +			config_param.cfg_main_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_main_param.bypass = 1;
> > +	}
> > +
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> > +		struct v4l2_format *rrzo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_param.cfg_resize_param.output.img_fmt =
> > +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.pixel_byte =
> > +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.size.w =
> > +			rrzo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_resize_param.output.size.h =
> > +			rrzo_fmt->fmt.pix_mp.height;
> > +		config_param.cfg_resize_param.output.size.stride =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_resize_param.output.size.xsize =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_resize_param.output.crop.left = 0x0;
> > +		config_param.cfg_resize_param.output.crop.top = 0x0;
> > +		config_param.cfg_resize_param.output.crop.width = sd_width;
> > +		config_param.cfg_resize_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "rrz out:%d:%d in:%d:%d",
> > +			  rrzo_fmt->fmt.pix_mp.width,
> > +			  rrzo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> > +			config_param.cfg_resize_param.output.pixel_byte,
> > +			config_param.cfg_resize_param.output.img_fmt);
> > +		dev_dbg(dev,
> > +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_resize_param.output.size.w,
> > +			config_param.cfg_resize_param.output.size.h,
> > +			config_param.cfg_resize_param.output.size.stride,
> > +			config_param.cfg_resize_param.output.size.xsize,
> > +			config_param.cfg_resize_param.output.crop.width,
> > +			config_param.cfg_resize_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = 1;
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> > +
> > +	isp_composer_hw_config(isp_ctx, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_port,
> > +		    struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct mtk_cam_dev_buffer **bundle_buffers;
> > +	unsigned int i, idx;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > +	memset(framejob, 0, sizeof(*framejob));
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	bundle_buffers = &frameparamsbase->buffers[0];
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> > +	framejob->request_fd = frameparamsbase->request_fd;
> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	idx = MTK_CAM_P1_META_IN_0;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.tuning_addr.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.tuning_addr.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Image output */
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[0].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "main stream address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[0].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Resize output */
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[1].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "packed out address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[1].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Meta output DMAs */
> > +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> > +		idx = MTK_CAM_P1_META_OUT_0 + i;
> > +		if (bundle_buffers[idx]) {
> > +			frameparams.meta_addrs[i].iova =
> > +			  bundle_buffers[idx]->daddr;
> > +			frameparams.meta_addrs[i].scp_addr =
> > +			  bundle_buffers[idx]->scp_addr;
> > +			list_add_tail(&bundle_buffers[idx]->list,
> > +				      &framejob->list_buf);
> > +		} else {
> > +			frameparams.meta_addrs[i].iova = 0;
> > +			frameparams.meta_addrs[i].scp_addr = 0;
> > +		}
> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		frameparamsbase->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> > +MODULE_LICENSE("GPL");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..6cf8bb4ba93a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,300 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-scp.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328U
> > +#define CAM_A_MAX_HEIGHT	2496U
> > +#define CAM_B_MAX_WIDTH		5376U
> > +#define CAM_B_MAX_HEIGHT	4032U
> > +
> > +#define CAM_MIN_WIDTH		80U
> > +#define CAM_MIN_HEIGHT		60U
> > +
> > +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_PDO		BIT(6)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_RSSO		BIT(9)
> > +#define R_PSO		BIT(10)
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define COMPOSRE_EVENT_BUF_SIZE		4
> > +
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +#define CQ_BUFFER_COUNT			3
> 
> Please align macro values using tabs.
> 
> > +
> > +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +enum {
> > +	img_fmt_unknown		= 0x0000,
> > +	img_fmt_raw_start	= 0x2200,
> > +	img_fmt_bayer8		= img_fmt_raw_start,
> > +	img_fmt_bayer10,
> > +	img_fmt_bayer12,
> > +	img_fmt_bayer14,
> > +	img_fmt_fg_bayer8,
> > +	img_fmt_fg_bayer10,
> > +	img_fmt_fg_bayer12,
> > +	img_fmt_fg_bayer14,
> > +};
> > +
> > +enum {
> > +	raw_pxl_id_b   = 0,
> > +	raw_pxl_id_gb,
> > +	raw_pxl_id_gr,
> > +	raw_pxl_id_r
> > +};
> > +
> > +enum {
> > +	default_pixel_mode = 0,
> > +	one_pixel_mode,
> > +	two_pixel_mode,
> > +	four_pixel_mode,
> > +	pixel_mode_num,
> > +};
> > +
> > +enum mtk_isp_scp_ipi_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_ipi_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> > +
> > +struct isp_clk_struct {
> > +	int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +};
> > +
> > +struct isp_device {
> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	int irq;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	struct mutex composer_tx_lock; /* isp composer work protection */
> > +
> > +	struct isp_thread composer_rx_thread;
> > +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> > +	atomic_t composer_evts_start;
> > +	atomic_t composer_evts_end;
> > +	spinlock_t composer_evts_lock; /* SCP events protection */
> > +	/* increase after ipi */
> > +	atomic_t ipi_occupied;
> > +	/* increase after frame enqueue */
> > +	atomic_t composing_frame;
> > +	/* current composed frame id */
> > +	atomic_t composed_frame_id;
> > +
> > +	struct isp_queue p1_enqueue_list;
> > +
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> > +
> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	struct sg_table sgtable;
> > +
> > +	/* increase after open, decrease when close */
> > +	atomic_t isp_user_cnt;
> > +	/* frame sequence number, increase per en-queue*/
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> > +	unsigned int enable_dma_ports;
> > +
> > +	void (*composer_deinit_donecb)(void *isp_ctx);
> > +
> > +	struct list_head list;
> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +
> > +	/* for SCP driver  */
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct isp_clk_struct isp_clk;
> 
> What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
> separate structs? They are only ever used in isp_p1_device.
> 

Ok, we will merge isp_clk_struct into mtk_isp_p1_ctx in next patch.

> > +	struct mtk_cam_dev *cam_dev;
> > +	struct isp_device *isp_devs;
> 
> Similarly, why are these allocated at runtime rather then just members
> of the struct?
> 

Ok, we will revise new isp_p1_device structure as below list:

struct isp_p1_device {
	struct platform_device *pdev;
	struct platform_device *scp_pdev;
	struct rproc *rproc_handle;
	struct mtk_isp_p1_ctx isp_ctx;
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
};


> In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
> afterwards the struct mtk_cam_dev and struct isp_devices are. There will
> only ever be ISP_DEV_NODE_NUM isp_devices.
> 
> Could this be changed to:
> 	struct mtk_cam_dev cam_dev;
> 	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> ?
> 

Yes, thanks for your suggestion as above.

> > +};
> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> > +}
> > +
> > +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			      unsigned int dma);
> > +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    struct p1_config_param *config_param);
> > +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> > +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    void (*donecb)(void *data));
> > +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> > +			  void *data,
> > +			  enum mtk_isp_scp_ipi_type type);
> 
> These functions are declared here, but implemented in mtk_cam-scp.c.
> Can the funtion declarations be moved to mtk_cam-scp.h?
> 

Ok, we will revise these function declared in next patch.

Best regards,


Jungo 

> > +
> > +/**
> > + * mtk_isp_open - open isp driver and initialize related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_open(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_release - release isp driver and related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_release(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_config - output image & meta data configuration.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_config(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> > + *
> > + * @dev:	isp device.
> > + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> > + *
> > + */
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase);
> > +
> > +/**
> > + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> > + * for non-per-frame DMA.
> > + *
> > + * @dev:	isp device.
> > + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> > + *
> > + */
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_idx,
> > +		    struct mtk_cam_dev_buffer *buffer);
> > +#endif /*__CAMERA_ISP_H*/
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, V2, 09/11] media: platform: Add Mediatek ISP P1 device driver
@ 2019-05-27 13:07       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-27 13:07 UTC (permalink / raw)
  To: Drew Davenport
  Cc: ryan.yu, Jerry-ch.Chen, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, frankie.chiu, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, sj.huang, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, shik, tfiga, christie.yu, zwisler

Hi, Drew:


On Fri, 2019-05-24 at 15:19 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:04AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> 
> A few comments inline.
> 

Appreciate your feedback on this patch set firstly.

> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  149 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1206 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  300 ++++
> >  3 files changed, 1655 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..342f0e0e9837
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,149 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT	BIT(0)
> > +#define CMOS_EN_BIT	BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST	BIT(0)
> > +#define HW_PASS1_DON_ST	BIT(11)
> > +#define SOF_INT_ST	BIT(12)
> > +#define SW_PASS1_DON_ST	BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST	BIT(4)
> > +#define TG_GBERR_ST	BIT(5)
> > +#define CQ_CODE_ERR_ST	BIT(6)
> > +#define CQ_APB_ERR_ST	BIT(7)
> > +#define CQ_VS_ERR_ST	BIT(8)
> > +#define AMX_ERR_ST	BIT(15)
> > +#define RMX_ERR_ST	BIT(16)
> > +#define BMX_ERR_ST	BIT(17)
> > +#define RRZO_ERR_ST	BIT(18)
> > +#define AFO_ERR_ST	BIT(19)
> > +#define IMGO_ERR_ST	BIT(20)
> > +#define AAO_ERR_ST	BIT(21)
> > +#define PSO_ERR_ST	BIT(22)
> > +#define LCSO_ERR_ST	BIT(23)
> > +#define BNR_ERR_ST	BIT(24)
> > +#define LSCI_ERR_ST	BIT(25)
> > +#define DMA_ERR_ST	BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST	BIT(4)
> > +#define AFO_DONE_ST	BIT(5)
> > +#define AAO_DONE_ST	BIT(7)
> > +#define PSO_DONE_ST	BIT(14)
> 
> Please align the values using tabs here and elsewhere.
> 

Ok, we will revise this coding style issue in next patch.

> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM	( \
> > +			VS_INT_ST |\
> > +			SOF_INT_ST |\
> > +			HW_PASS1_DON_ST |\
> > +			SW_PASS1_DON_ST)
> > +
> > +/* IRQ Warning Mask */
> > +#define INT_ST_MASK_CAM_WARN	(\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				PSO_ERR_ST | \
> > +				LCSO_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR	(\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				CQ_CODE_ERR_ST |\
> > +				CQ_APB_ERR_ST |\
> > +				CQ_VS_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				RMX_ERR_ST |\
> > +				BMX_ERR_ST |\
> > +				BNR_ERR_ST |\
> > +				LSCI_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM	(\
> > +				SOF_INT_ST |\
> > +				SW_PASS1_DON_ST |\
> > +				VS_INT_ST |\
> > +				TG_ERR_ST |\
> > +				TG_GBERR_ST |\
> > +				RRZO_ERR_ST |\
> > +				AFO_ERR_ST |\
> > +				IMGO_ERR_ST |\
> > +				AAO_ERR_ST |\
> > +				DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM	(\
> > +			AFO_DONE_ST |\
> > +			AAO_DONE_ST |\
> > +			PSO_DONE_ST |\
> > +			FLKO_DONE_ST)
> > +
> > +/* Status check */
> > +#define REG_CTL_EN		0x0004
> > +#define REG_CTL_DMA_EN		0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2		0x0018
> > +#define REG_CTL_RAW_INT_EN	0x0020
> > +#define REG_CTL_RAW_INT_STAT	0x0024
> > +#define REG_CTL_RAW_INT2_STAT	0x0034
> > +#define REG_CTL_RAW_INT3_STAT	0x00c4
> > +#define REG_CTL_TWIN_STAT	0x0050
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_SEN_GRAB_PIX	0x0238
> > +#define REG_TG_SEN_GRAB_LIN	0x023c
> > +#define REG_TG_VF_CON		0x0234
> > +#define REG_TG_SUB_PERIOD	0x02a4
> > +
> > +#define REG_IMGO_BASE_ADDR	0x1020
> > +#define REG_RRZO_BASE_ADDR	0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT	0x1360
> > +#define REG_RRZO_ERR_STAT	0x1364
> > +#define REG_AAO_ERR_STAT	0x1368
> > +#define REG_AFO_ERR_STAT	0x136c
> > +#define REG_LCSO_ERR_STAT	0x1370
> > +#define REG_UFEO_ERR_STAT	0x1374
> > +#define REG_PDO_ERR_STAT	0x1378
> > +#define REG_BPCI_ERR_STAT	0x137c
> > +#define REG_LSCI_ERR_STAT	0x1384
> > +#define REG_PDI_ERR_STAT	0x138c
> > +#define REG_LMVO_ERR_STAT	0x1390
> > +#define REG_FLKO_ERR_STAT	0x1394
> > +#define REG_PSO_ERR_STAT	0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR	0x0198
> > +#define REG_HW_FRAME_NUM	0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX	0x14dc
> > +#define REG_META1_VB2_INDEX	0x151c
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..fc874ec8f7f0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1206 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +/* list of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"CAMSYS_CAM_CGPDN", "CAMSYS_CAMTG_CGPDN"
> > +};
> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_isp_notify(struct mtk_isp_p1_ctx *isp_ctx,
> > +			   unsigned int request_fd,
> > +			   unsigned int frame_seq_no,
> > +			   struct list_head *list_buf,
> > +			   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev_finish_param fram_param;
> > +
> > +	fram_param.list_buf = list_buf;
> > +	fram_param.request_fd = request_fd;
> > +	fram_param.frame_seq_no = frame_seq_no;
> > +	fram_param.state = state;
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d\n",
> > +		fram_param.request_fd,
> > +		fram_param.frame_seq_no);
> > +	mtk_cam_dev_job_finish(p1_dev->cam_dev, &fram_param);
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct vb2_queue *vb2_queue = &cam_dev->mem2mem2_nodes[node_id].vbq;
> > +	struct vb2_buffer *vb;
> > +	struct vb2_v4l2_buffer *vbb;
> > +
> > +	if (!cam_dev->mem2mem2_nodes[node_id].enabled)
> > +		return;
> > +
> > +	mutex_lock(vb2_queue->lock);
> > +	list_for_each_entry(vb, &vb2_queue->queued_list, queued_entry) {
> > +		vbb = to_vb2_v4l2_buffer(vb);
> > +		if (vbb->request_fd < 0 &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vbb->vb2_buf.timestamp = ktime_get_ns();
> > +			vbb->sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +		}
> > +	}
> > +	mutex_unlock(vb2_queue->lock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_isp_notify(isp_ctx,
> > +				       framejob->request_fd,
> > +				       framejob->frame_seq_no,
> > +				       &framejob->list_buf,
> > +				       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		if (ret == ERESTARTSYS) {
> > +			dev_err(dev, "interrupted by a signal!\n");
> > +			continue;
> > +		}
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		memset(&isp_ctx->irq_event_datas[i], 0x00, sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & VS_INT_ST) {
> > +			/* Notify specific HW events to user space */
> > +			mtk_cam_dev_event_frame_sync(cam_dev,
> > +						     event_data.frame_seq_no);
> > +			dev_dbg(dev,
> > +				"event IRQ:0x%x DMA:0x%x is sent\n",
> > +				event_data.irq_status_mask,
> > +				event_data.dma_status_mask);
> > +		}
> > +
> > +		if (event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int cq_addr_index;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	if (cq_num > frame_num) {
> > +		cq_addr_index = frame_num % CQ_BUFFER_COUNT;
> > +
> > +		writel(base_addr +
> > +			(dma_addr_t)(CQ_ADDRESS_OFFSET * cq_addr_index),
> > +			isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:%d",
> > +			cq_num, frame_num, cq_addr_index);
> > +	} else {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +	}
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	return cq_num;
> > +}
> > +
> > +static int irq_handle_notify_event(struct isp_device *isp_dev,
> > +				   unsigned int irqstatus,
> > +				   unsigned int dmastatus)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask |=
> > +		(irqstatus & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask |=
> > +		(dmastatus & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +
> > +	dev_dbg(isp_dev->dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irqstatus & INT_ST_MASK_CAM),
> > +		(dmastatus & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +
> > +	return 0;
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cardinalnum, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irqstatus, errstatus, warnstatus, dmastatus;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev->streaming)
> > +		return IRQ_HANDLED;
> > +
> > +	cardinalnum = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irqstatus = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dmastatus = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (irqstatus == 0)
> > +		return IRQ_HANDLED;
> > +
> > +	errstatus = irqstatus & INT_ST_MASK_CAM_ERR;
> > +	warnstatus = irqstatus & INT_ST_MASK_CAM_WARN;
> > +	irqstatus = irqstatus & INT_ST_MASK_CAM;
> > +
> > +	/* sof , done order check . */
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	if ((irqstatus & HW_PASS1_DON_ST) && (irqstatus & SOF_INT_ST)) {
> > +		dev_warn(dev,
> > +			 "isp sof_don block, sof_cnt:%d\n",
> > +			 isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue ready frame */
> > +		irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +		isp_dev->current_frame = hw_frame_num;
> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irqstatus & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +
> > +		if ((irqstatus & INT_ST_MASK_CAM) ||
> > +		    (dmastatus & DMA_ST_MASK_CAM))
> > +			irq_handle_notify_event(isp_dev, irqstatus, dmastatus);
> > +	}
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	if (irqstatus & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	if (irqstatus & SW_PASS1_DON_ST) {
> > +		int num = atomic_dec_return(&isp_ctx->composing_frame);
> > +
> > +		dev_dbg(dev, "SW_PASS1_DON_ST queued frame:%d\n", num);
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible
> > +				(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	/* check ISP error status */
> > +	if (errstatus) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x/0x%x\n",
> > +			irqstatus, warnstatus, errstatus);
> > +
> > +		/* show DMA errors in detail */
> > +		if (errstatus & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irqstatus & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> > +			'A' + cardinalnum,
> > +			isp_dev->sof_count,
> > +			irqstatus,
> > +			dmastatus,
> > +			hw_frame_num,
> > +			cq_num);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_clk.num_clks,
> > +				      p1_dev->isp_clk.clk_list);
> > +	if (ret < 0)
> > +		goto clk_err;
> > +	return 0;
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	dev_info(dev, "- %s dev id:%d\n", __func__, dev->id);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_clk.num_clks,
> > +				   p1_dev->isp_clk.clk_list);
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	/* Allocate context */
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> > +
> > +	p1_dev->isp_devs =
> > +		devm_kzalloc(dev,
> > +			     sizeof(struct isp_device) * ISP_DEV_NODE_NUM,
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->cam_dev =
> > +		devm_kzalloc(dev, sizeof(struct mtk_cam_dev), GFP_KERNEL);
> > +	if (!p1_dev->isp_devs)
> > +		return -ENOMEM;
> > +
> > +	/* iomap registers */
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_info(dev, "cam%u, map_addr=0x%lx\n",
> > +			 i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* reg & interrupts index is shifted with 1  */
> > +			isp_dev->irq = platform_get_irq(pdev, i - 1);
> > +			if (isp_dev->irq > 0) {
> > +				ret = devm_request_irq(dev, isp_dev->irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> > +						       (void *)isp_dev);
> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						isp_dev->irq);
> > +					return ret;
> > +				}
> > +				dev_info(dev, "Registered irq=%d, ISR:%s\n",
> > +					 isp_dev->irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> > +
> > +	p1_dev->isp_clk.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_clk.clk_list =
> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_clk.num_clks,
> > +			     sizeof(*p1_dev->isp_clk.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_clk.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_clk.num_clks; ++i)
> > +		p1_dev->isp_clk.clk_list->id = mtk_isp_clks[i];
> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_clk.num_clks,
> > +				p1_dev->isp_clk.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory\n");
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, p1_dev->cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	atomic_set(&p1_dev->isp_ctx.isp_user_cnt, 0);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int usercount, module;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* disable VF */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, The TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct isp_device *isp_dev;
> > +	unsigned int reg_val;
> > +	int module, usercount;
> > +
> > +	module = p1_dev->isp_ctx.isp_hw_module;
> > +	usercount = atomic_read(&p1_dev->isp_ctx.isp_user_cnt);
> > +
> > +	dev_dbg(dev, "- %s:%d\n", __func__, usercount);
> > +
> > +	/* If no user count, no further action */
> > +	if (!usercount)
> > +		return 0;
> > +
> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev->streaming) {
> > +		isp_dev = &p1_dev->isp_devs[module];
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n",
> > +		p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret < 0) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		mutex_init(&isp_ctx->composer_tx_lock);
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +	return 0;
> > +}
> > +
> > +static int isp_uninit_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	atomic_set(&isp_ctx->isp_user_cnt, 0);
> > +
> > +	if (!IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enable_dma_ports(struct mtk_cam_dev *cam_dev)
> 
> I think s/enable/enabled would be a clearer name for both the function
> and local variable.
> 

Ok, we will revised this "enabled" working in next patch.

> > +{
> > +	unsigned int enable_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enable_dma_ports = 0;
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		if (cam_dev->mem2mem2_nodes[i].enabled)
> > +			enable_dma_ports |=
> > +				cam_dev->mem2mem2_nodes[i].desc.dma_port;
> > +	}
> > +	dev_dbg(&cam_dev->pdev->dev, "%s enable_dma_ports:0x%x",
> > +		__func__, enable_dma_ports);
> > +
> > +	return enable_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return raw_pxl_id_b;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return raw_pxl_id_gb;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return raw_pxl_id_gr;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return raw_pxl_id_r;
> > +	default:
> > +		return raw_pxl_id_b;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return img_fmt_bayer8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return img_fmt_bayer10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return img_fmt_bayer12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return img_fmt_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return img_fmt_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return img_fmt_fg_bayer8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return img_fmt_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return img_fmt_fg_bayer10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return img_fmt_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return img_fmt_fg_bayer12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return img_fmt_bayer14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return img_fmt_fg_bayer14;
> > +	default:
> > +		return img_fmt_unknown;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 10;
> > +	}
> > +}
> > +
> > +static void composer_deinit_done_cb(void *data)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +
> > +	disable_sys_clock(p1_dev);
> > +	/* Notify PM */
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_open(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	s32 usercount = atomic_inc_return(&isp_ctx->isp_user_cnt);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__, usercount);
> > +
> > +	if (usercount == 1) {
> > +		ret = isp_setup_scp_rproc(p1_dev);
> > +		if (ret)
> > +			goto scp_err;
> > +
> > +		/* ISP HW INIT */
> > +		isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> > +		/* Use pure RAW as default HW path */
> > +		isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +		/* Check enabled DMAs which is configured by media setup */
> > +		isp_ctx->enable_dma_ports =
> > +			get_enable_dma_ports(p1_dev->cam_dev);
> > +
> > +		if (!isp_ctx->enable_dma_ports) {
> > +			dev_dbg(dev, "No DMAs are enabled\n");
> > +			ret = -EINVAL;
> > +			goto scp_err;
> > +		}
> > +
> > +		pm_runtime_get_sync(dev);
> > +
> > +		ret = isp_init_context(p1_dev);
> > +		if (ret)
> > +			goto ctx_err;
> > +		ret = isp_composer_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +		ret = isp_composer_hw_init(isp_ctx);
> > +		if (ret)
> > +			goto composer_err;
> > +
> > +		isp_composer_meta_config(&p1_dev->isp_ctx,
> > +					 isp_ctx->enable_dma_ports);
> > +	}
> > +
> > +	return 0;
> > +composer_err:
> > +	isp_uninit_context(p1_dev);
> > +ctx_err:
> > +	pm_runtime_put_sync(dev);
> > +scp_err:
> > +	atomic_dec_return(&isp_ctx->isp_user_cnt);
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_release(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	if (atomic_dec_and_test(&p1_dev->isp_ctx.isp_user_cnt)) {
> > +		isp_composer_hw_deinit(isp_ctx, composer_deinit_done_cb);
> > +		isp_uninit_context(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "%s usercount=%d\n", __func__,
> > +		atomic_read(&p1_dev->isp_ctx.isp_user_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_format;
> > +	unsigned int sd_width, sd_height;
> > +	unsigned int enable_dma_ports, idx;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enable_dma_ports = isp_ctx->enable_dma_ports;
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enable_dma_ports);
> > +
> > +	/* sensor config */
> > +	sd_format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor,
> > +			       pad, get_fmt, NULL, &sd_format);
> > +
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor:%s g_fmt on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"sensor get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_format.format.width, sd_format.format.height,
> > +		sd_format.format.code, sd_format.format.field,
> > +		sd_format.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = one_pixel_mode;
> > +	/* support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_format.format.code);
> > +	config_param.cfg_in_param.img_fmt =
> > +		get_sensor_fmt(sd_format.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_format.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_format.format.height;
> > +	sd_width = sd_format.format.width;
> > +	sd_height = sd_format.format.height;
> > +
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> 
> The idx variable is unnecessary. Just use MTK_CAM_P1_... to index into
> mem2mem2_nodes directly here and below.
> 

Ok, we will remove idx variable and use const value to index.

> > +	if ((enable_dma_ports & R_IMGO) == R_IMGO) {
> > +		struct v4l2_format *imgo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_param.cfg_main_param.bypass = 0;
> > +
> > +		config_param.cfg_main_param.output.img_fmt =
> > +			get_img_fmt(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.pixel_byte =
> > +			get_pixel_byte(imgo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_main_param.output.size.w =
> > +			imgo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_main_param.output.size.h =
> > +			imgo_fmt->fmt.pix_mp.height;
> > +
> > +		config_param.cfg_main_param.output.size.stride =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_main_param.output.size.xsize =
> > +			imgo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_main_param.output.crop.left = 0x0;
> > +		config_param.cfg_main_param.output.crop.top = 0x0;
> > +
> > +		config_param.cfg_main_param.output.crop.width = sd_width;
> > +		config_param.cfg_main_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(imgo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  imgo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "img out:%d:%d in:%d:%d",
> > +			  imgo_fmt->fmt.pix_mp.width,
> > +			  imgo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev,
> > +			"imgo pixel_byte:%d img_fmt:0x%x raw:%d\n",
> > +			config_param.cfg_main_param.output.pixel_byte,
> > +			config_param.cfg_main_param.output.img_fmt,
> > +			config_param.cfg_main_param.pure_raw);
> > +		dev_dbg(dev,
> > +			"imgo param:size=%0dx%0d, stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_main_param.output.size.w,
> > +			config_param.cfg_main_param.output.size.h,
> > +			config_param.cfg_main_param.output.size.stride,
> > +			config_param.cfg_main_param.output.size.xsize,
> > +			config_param.cfg_main_param.output.crop.width,
> > +			config_param.cfg_main_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_main_param.bypass = 1;
> > +	}
> > +
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if ((enable_dma_ports & R_RRZO) == R_RRZO) {
> > +		struct v4l2_format *rrzo_fmt =
> > +			&p1_dev->cam_dev->mem2mem2_nodes[idx].vdev_fmt;
> > +
> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_param.cfg_resize_param.output.img_fmt =
> > +			get_img_fmt(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.pixel_byte =
> > +			get_pixel_byte(rrzo_fmt->fmt.pix_mp.pixelformat);
> > +		config_param.cfg_resize_param.output.size.w =
> > +			rrzo_fmt->fmt.pix_mp.width;
> > +		config_param.cfg_resize_param.output.size.h =
> > +			rrzo_fmt->fmt.pix_mp.height;
> > +		config_param.cfg_resize_param.output.size.stride =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +		config_param.cfg_resize_param.output.size.xsize =
> > +			rrzo_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +		config_param.cfg_resize_param.output.crop.left = 0x0;
> > +		config_param.cfg_resize_param.output.crop.top = 0x0;
> > +		config_param.cfg_resize_param.output.crop.width = sd_width;
> > +		config_param.cfg_resize_param.output.crop.height = sd_height;
> > +
> > +		WARN_ONCE(rrzo_fmt->fmt.pix_mp.width > sd_width ||
> > +			  rrzo_fmt->fmt.pix_mp.height > sd_height,
> > +			  "rrz out:%d:%d in:%d:%d",
> > +			  rrzo_fmt->fmt.pix_mp.width,
> > +			  rrzo_fmt->fmt.pix_mp.height,
> > +			  sd_width,
> > +			  sd_height);
> > +
> > +		dev_dbg(dev, "rrzo pixel_byte:%d img_fmt:0x%x\n",
> > +			config_param.cfg_resize_param.output.pixel_byte,
> > +			config_param.cfg_resize_param.output.img_fmt);
> > +		dev_dbg(dev,
> > +			"rrzo param:size=%0dx%0d,stride:%d,xsize:%d,crop=%0dx%0d\n",
> > +			config_param.cfg_resize_param.output.size.w,
> > +			config_param.cfg_resize_param.output.size.h,
> > +			config_param.cfg_resize_param.output.size.stride,
> > +			config_param.cfg_resize_param.output.size.xsize,
> > +			config_param.cfg_resize_param.output.crop.width,
> > +			config_param.cfg_resize_param.output.crop.height);
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = 1;
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enable_dma_ports;
> > +
> > +	isp_composer_hw_config(isp_ctx, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_port,
> > +		    struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_CMD);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct mtk_cam_dev_buffer **bundle_buffers;
> > +	unsigned int i, idx;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > +	memset(framejob, 0, sizeof(*framejob));
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	bundle_buffers = &frameparamsbase->buffers[0];
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> > +	framejob->request_fd = frameparamsbase->request_fd;
> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	idx = MTK_CAM_P1_META_IN_0;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.tuning_addr.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.tuning_addr.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Image output */
> > +	idx = MTK_CAM_P1_MAIN_STREAM_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[0].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[0].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "main stream address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[0].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Resize output */
> > +	idx = MTK_CAM_P1_PACKED_BIN_OUT;
> > +	if (bundle_buffers[idx]) {
> > +		frameparams.img_dma_buffers[1].buffer.iova =
> > +			bundle_buffers[idx]->daddr;
> > +		frameparams.img_dma_buffers[1].buffer.scp_addr =
> > +			bundle_buffers[idx]->scp_addr;
> > +		dev_dbg(dev, "packed out address iova:0x%x\n",
> > +			frameparams.img_dma_buffers[1].buffer.iova);
> > +		list_add_tail(&bundle_buffers[idx]->list,
> > +			      &framejob->list_buf);
> > +	}
> > +
> > +	/* Meta output DMAs */
> > +	for (i = 0; i < MAX_META_DMA_NODES; i++) {
> > +		idx = MTK_CAM_P1_META_OUT_0 + i;
> > +		if (bundle_buffers[idx]) {
> > +			frameparams.meta_addrs[i].iova =
> > +			  bundle_buffers[idx]->daddr;
> > +			frameparams.meta_addrs[i].scp_addr =
> > +			  bundle_buffers[idx]->scp_addr;
> > +			list_add_tail(&bundle_buffers[idx]->list,
> > +				      &framejob->list_buf);
> > +		} else {
> > +			frameparams.meta_addrs[i].iova = 0;
> > +			frameparams.meta_addrs[i].scp_addr = 0;
> > +		}
> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(isp_ctx, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		frameparamsbase->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> > +MODULE_LICENSE("GPL");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..6cf8bb4ba93a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,300 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-scp.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328U
> > +#define CAM_A_MAX_HEIGHT	2496U
> > +#define CAM_B_MAX_WIDTH		5376U
> > +#define CAM_B_MAX_HEIGHT	4032U
> > +
> > +#define CAM_MIN_WIDTH		80U
> > +#define CAM_MIN_HEIGHT		60U
> > +
> > +#define IMG_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH		CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT		CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH		CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT		CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_PDO		BIT(6)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_RSSO		BIT(9)
> > +#define R_PSO		BIT(10)
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define COMPOSRE_EVENT_BUF_SIZE		4
> > +
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +#define CQ_BUFFER_COUNT			3
> 
> Please align macro values using tabs.
> 
> > +
> > +#define IRQ_STAT_STR "cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +enum {
> > +	img_fmt_unknown		= 0x0000,
> > +	img_fmt_raw_start	= 0x2200,
> > +	img_fmt_bayer8		= img_fmt_raw_start,
> > +	img_fmt_bayer10,
> > +	img_fmt_bayer12,
> > +	img_fmt_bayer14,
> > +	img_fmt_fg_bayer8,
> > +	img_fmt_fg_bayer10,
> > +	img_fmt_fg_bayer12,
> > +	img_fmt_fg_bayer14,
> > +};
> > +
> > +enum {
> > +	raw_pxl_id_b   = 0,
> > +	raw_pxl_id_gb,
> > +	raw_pxl_id_gr,
> > +	raw_pxl_id_r
> > +};
> > +
> > +enum {
> > +	default_pixel_mode = 0,
> > +	one_pixel_mode,
> > +	two_pixel_mode,
> > +	four_pixel_mode,
> > +	pixel_mode_num,
> > +};
> > +
> > +enum mtk_isp_scp_ipi_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_ipi_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> > +
> > +struct isp_clk_struct {
> > +	int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +};
> > +
> > +struct isp_device {
> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	int irq;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	struct mutex composer_tx_lock; /* isp composer work protection */
> > +
> > +	struct isp_thread composer_rx_thread;
> > +	struct mtk_isp_scp_p1_cmd composer_evts[COMPOSRE_EVENT_BUF_SIZE];
> > +	atomic_t composer_evts_start;
> > +	atomic_t composer_evts_end;
> > +	spinlock_t composer_evts_lock; /* SCP events protection */
> > +	/* increase after ipi */
> > +	atomic_t ipi_occupied;
> > +	/* increase after frame enqueue */
> > +	atomic_t composing_frame;
> > +	/* current composed frame id */
> > +	atomic_t composed_frame_id;
> > +
> > +	struct isp_queue p1_enqueue_list;
> > +
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> > +
> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	struct sg_table sgtable;
> > +
> > +	/* increase after open, decrease when close */
> > +	atomic_t isp_user_cnt;
> > +	/* frame sequence number, increase per en-queue*/
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> > +	unsigned int enable_dma_ports;
> > +
> > +	void (*composer_deinit_donecb)(void *isp_ctx);
> > +
> > +	struct list_head list;
> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +
> > +	/* for SCP driver  */
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct isp_clk_struct isp_clk;
> 
> What's the benefit of having mtk_isp_p1_ctx and isp_clk_struct in
> separate structs? They are only ever used in isp_p1_device.
> 

Ok, we will merge isp_clk_struct into mtk_isp_p1_ctx in next patch.

> > +	struct mtk_cam_dev *cam_dev;
> > +	struct isp_device *isp_devs;
> 
> Similarly, why are these allocated at runtime rather then just members
> of the struct?
> 

Ok, we will revise new isp_p1_device structure as below list:

struct isp_p1_device {
	struct platform_device *pdev;
	struct platform_device *scp_pdev;
	struct rproc *rproc_handle;
	struct mtk_isp_p1_ctx isp_ctx;
	struct mtk_cam_dev cam_dev;
	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
};


> In mtk_isp_probe the struct isp_p1_device is allocated, and immediately
> afterwards the struct mtk_cam_dev and struct isp_devices are. There will
> only ever be ISP_DEV_NODE_NUM isp_devices.
> 
> Could this be changed to:
> 	struct mtk_cam_dev cam_dev;
> 	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> ?
> 

Yes, thanks for your suggestion as above.

> > +};
> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> > +}
> > +
> > +int isp_composer_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +int isp_composer_hw_init(struct mtk_isp_p1_ctx *isp_ctx);
> > +void isp_composer_meta_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			      unsigned int dma);
> > +void isp_composer_hw_config(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    struct p1_config_param *config_param);
> > +void isp_composer_stream(struct mtk_isp_p1_ctx *isp_ctx, int on);
> > +void isp_composer_hw_deinit(struct mtk_isp_p1_ctx *isp_ctx,
> > +			    void (*donecb)(void *data));
> > +void isp_composer_enqueue(struct mtk_isp_p1_ctx *isp_ctx,
> > +			  void *data,
> > +			  enum mtk_isp_scp_ipi_type type);
> 
> These functions are declared here, but implemented in mtk_cam-scp.c.
> Can the funtion declarations be moved to mtk_cam-scp.h?
> 

Ok, we will revise these function declared in next patch.

Best regards,


Jungo 

> > +
> > +/**
> > + * mtk_isp_open - open isp driver and initialize related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_open(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_release - release isp driver and related resources.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_release(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_config - output image & meta data configuration.
> > + *
> > + * @dev:	isp device.
> > + *
> > + */
> > +int mtk_isp_config(struct device *dev);
> > +
> > +/**
> > + * mtk_isp_req_enqueue - enqueue a frame bundle (per-frame basis) to ISP driver.
> > + *
> > + * @dev:	isp device.
> > + * @frameparamsbase: pointer to &struct mtk_cam_dev_start_param.
> > + *
> > + */
> > +int mtk_isp_req_enqueue(struct device *dev,
> > +			struct mtk_cam_dev_start_param *frameparamsbase);
> > +
> > +/**
> > + * mtk_isp_enqueue - enqueue a single frame to ISP driver
> > + * for non-per-frame DMA.
> > + *
> > + * @dev:	isp device.
> > + * @buffer: pointer to &struct mtk_cam_dev_buffer.
> > + *
> > + */
> > +int mtk_isp_enqueue(struct device *dev,
> > +		    unsigned int dma_idx,
> > +		    struct mtk_cam_dev_buffer *buffer);
> > +#endif /*__CAMERA_ISP_H*/
> > -- 
> > 2.18.0
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-05-24 18:49       ` Drew Davenport
  (?)
@ 2019-05-28  1:00         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-28  1:00 UTC (permalink / raw)
  To: Drew Davenport
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi, Drew:

Appreciate your feedbacks on this patch set firstly.

On Fri, 2019-05-24 at 12:49 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and six video devices during
> > initialization. Moreover, it also connects with sensor and
> > senif drivers with V4L2 async APIs.
> 
> Thanks for the patch. I've made a few comments inline. As a general
> comment, what do you think of merging mtk_cam-dev.c and
> mtk_cam-v4l2-util.c into one file? They seem to call into one another
> and I'm not sure how beneficial it is to have them separate.
> 
> I have some comments on the other patches in this series that came about
> while I was reviewing this, which I will send as well.
> 
> [snip]
>  

Ok, we will merge tk_cam-dev.c into mtk_cam-v4l2-util.c in next patch
set.

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..5a581ab65945
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,19 @@
> > +#
> > +# Copyright (C) 2018 MediaTek Inc.
> > +#
> > +# This program is free software: you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License version 2 as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +
> > +mtk-cam-isp-objs += \
> > +	mtk_cam.o mtk_cam-dev.o \
> > +	mtk_cam-ctrl.o mtk_cam-scp.o \
> > +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
> 
> Some of these files are added in other patches. Consider adding files to
> the Makefile in the same patch a file is added.
> 

Ok, we will add the corresponding files in each patches.

> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > new file mode 100644
> > index 000000000000..dda8a7b161ee
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > @@ -0,0 +1,758 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + * Copyright (C) 2017 Google, Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/of.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-smem.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
> 
> This is not used anywhere. Please remove.
> 

Ok, we will remove unused variable.

> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> > +static struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct mtk_cam_dev_queues_setting queues_setting = {
> > +	.output_node_descs = output_queues,
> > +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> > +	.capture_node_descs = capture_queues,
> > +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> > +};
> 
> I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup
> 

Ok, we will remove mtk_cam_dev_queue_setup structure and revise
mtk_cam_dev_queue_setup function.

> > +
> > +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
> 
> Since only one_pixel_mode is supported, this function can be removed and
> the callsite replaced with ALIGN(size, 2). This function can be added
> once more when other pixel modes are supported.
> 

Got it, we will align_main_stream_size & align_packetd_out_size
functions and remove pix_mode argument in cal_main_stream_stride &
functions.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 4);
> > +	case one_pixel_mode:
> > +		return ALIGN(size, 2);
> > +	default:
> > +		break;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static unsigned int align_packetd_out_size(__u32 size,
> > +					   unsigned int pix_mode,
> > +					   __u32 fmt)
> 
> This is only ever called with one_pixel_mode. Remove the pix_mode
> argument and unreachable code.
> 

Ditto.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 16);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case one_pixel_mode:
> > +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> > +			return ALIGN(size, 4);
> > +		else
> > +			return ALIGN(size, 8);
> > +	default:
> > +		return ALIGN(size, 16);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static __u32 cal_main_stream_stride(struct device *dev,
> > +				    __u32 width,
> > +				    __u32 pix_fmt,
> > +				    __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_main_stream_size(stride, pix_mode);
> > +
> > +	dev_dbg(dev,
> > +		"main width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_packed_out_stride(struct device *dev,
> > +				   __u32 width,
> > +				   __u32 pix_fmt,
> > +				   __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = DIV_ROUND_UP(width * 3, 2);
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> > +
> > +	dev_dbg(dev,
> > +		"packed width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_img_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> > +					     one_pixel_mode);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> > +					    one_pixel_mode);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* The helper to configure the device context */
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting)
> 
> This is only ever called with the same mtk_cam_dev_queues_setting
> struct. I think you can remove that struct altogether and just set the
> mtk_cam_dev_node_desc* for each node from output_queues and
> capture_queues directly.
> 
> Also this can be a static function.
> 

Thanks for your suggestion.
We will revise in next patch as below.

/* The helper to configure the device context */
static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)

> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < setting->total_output_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->output_node_descs[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < setting->total_capture_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->capture_node_descs[i];
> > +
> > +	cam_dev->dev_node_num = node_idx;
> 
> This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
> MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
> that instead of dev_node_num?
> 

Ok, we will use new MTK_CAM_P1_TOTAL_NODES const value to replace
dev_node_num in next patch.

> > +}
> > +
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *fram_param)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +
> > +	if (!cam_dev->streaming)
> > +		return 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev,
> > +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> > +		fram_param->request_fd,
> > +		fram_param->frame_seq_no,
> > +		fram_param->state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> > +		buf->vbb.sequence = fram_param->frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +
> > +	return 0;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = cal_img_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> > +}
> > +
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> > +}
> > +
> > +/* Get the default format setting */
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue_desc,
> > +				  struct v4l2_format *dest)
> > +{
> > +	struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_fmt_set_img(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +/* Get a free buffer from a video node */
> > +static struct mtk_cam_dev_buffer *
> > +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> > +{
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct mtk_cam_video_device *vdev;
> > +
> > +	if (node > cam_dev->dev_node_num || node < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> > +		return NULL;
> > +	}
> > +	vdev = &cam_dev->mem2mem2_nodes[node];
> > +
> > +	spin_lock(&vdev->slock);
> > +	buf = list_first_entry_or_null(&vdev->pending_list,
> > +				       struct mtk_cam_dev_buffer,
> > +				       list);
> > +	if (!buf) {
> > +		spin_unlock(&vdev->slock);
> > +		return NULL;
> > +	}
> > +	list_del(&buf->list);
> > +	spin_unlock(&vdev->slock);
> 
> Can this be simplified by going:
> spin_lock();
> buf = list_first_entry_or_null(...);
> if (buf) list_del(...);
> spin_unlock();
> return buf;
> 

Ok, we will simplify this implementation as you suggested.

> > +
> > +	return buf;
> > +}
> > +
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
> 
> This only ever returns 0, so make it a void function.
> 

Ok, fix in next patch.

> > +{
> > +	unsigned int node;
> > +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_start_param s_param;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> > +		return 0;
> > +	}
> > +
> > +	/* Check all enabled nodes to collect its buffer  */
> > +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> > +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> > +			continue;
> > +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> > +		if (!buf)
> > +			continue;
> > +
> > +		/* TBD: use buf_init callback function */
> > +		buf->daddr =
> > +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> > +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> > +				cam_dev->smem_dev, buf->daddr);
> > +		} else {
> > +			buf->scp_addr = 0;
> > +		}
> > +
> > +		dev_dbg(dev,
> > +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> > +			node,
> > +			buf->vbb.request_fd,
> > +			buf->vbb.vb2_buf.index,
> > +			buf->vbb.vb2_buf.state,
> > +			&buf->daddr,
> > +			&buf->scp_addr);
> > +
> > +		s_param.buffers[node] = buf;
> > +		s_param.request_fd = buf->vbb.request_fd;
> > +	}
> > +
> > +	/* Trigger en-queued job to driver */
> > +	mtk_isp_req_enqueue(dev, &s_param);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> > +
> > +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> > +
> > +	/* v4l2 sub-device registration */
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> > +
> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret)
> 
> If this fails do we need to undo the stuff done in
> mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will call mtk_cam_v4l2_unregister(cam_dev) if failed in
mtk_cam_v4l2_async_register function in next patch.


> > +		return ret;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > new file mode 100644
> > index 000000000000..410460de44fa
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > @@ -0,0 +1,250 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_H__
> > +#define __MTK_CAM_DEV_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> > +
> > +#define MTK_CAM_DEV_NODES		11
> > +
> > +#define MTK_CAM_P1_META_IN_0		0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> > +#define MTK_CAM_P1_META_OUT_0		3
> > +#define MTK_CAM_P1_META_OUT_1		4
> > +#define MTK_CAM_P1_META_OUT_2		5
> > +#define MTK_CAM_P1_META_OUT_3		6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE	6
> 
> Please align macro values using tabs.
> 

Ok, we will fix this coding style issue in next patch.

> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +};
> > +
> > +/* Attributes setup by device owner */
> > +struct mtk_cam_dev_queues_setting {
> > +	const struct mtk_cam_dev_node_desc *output_node_descs;
> > +	unsigned int total_output_nodes;
> > +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> > +	unsigned int total_capture_nodes;
> > +};
> > +
> > +struct mtk_cam_dev_start_param {
> > +	int request_fd;
> > +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> > +};
> > +
> > +struct mtk_cam_dev_finish_param {
> > +	int request_fd;
> > +	unsigned int frame_seq_no;
> > +	unsigned int state;
> > +	struct list_head *list_buf;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	struct v4l2_format *fmts;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct video_device vdev;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> > +	struct mutex lock;
> > +	/* protect for pending_list */
> > +	spinlock_t slock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @dev_node_num: The number of supported V4L2 video device nodes
> > + * @request_fd:	The file descriptor of request API
> > + * @request_count: The buffer count of request API
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int dev_node_num;
> > +	int request_fd;
> > +	unsigned int request_count;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler);
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting);
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *param);
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no);
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id);
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue,
> > +				  struct v4l2_format *dest_fmt);
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id);
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_dev *
> > +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> > +{
> > +	return container_of(__sd,
> > +		struct mtk_cam_dev, subdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> > +{
> > +	return container_of(__vq,
> > +		struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb,
> > +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +#endif /* __MTK_CAM_DEV_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > new file mode 100644
> > index 000000000000..196aaef3d854
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > @@ -0,0 +1,1086 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define MTK_CAM_SENINF_PAD_SRC			4
> > +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> > +
> > +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> > +			       struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	cam_dev->request_fd = -1;
> > +	cam_dev->request_count = 0;
> > +
> > +	return mtk_isp_open(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> > +				struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	return mtk_isp_release(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
> 
> "get" implies that this function will retrieve something without
> side effects, which is not the case here. In the error case, the return
> value is ignored by the caller as well.
> 
> Consider making this function return a struct v4l2_subdev* (or NULL in
> the error case) and let the caller set mtk_cam_dev::sensor.
> 

Ok, below is the new function prototype.
static struct v4l2_subdev *
mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)

> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	cam_dev->sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count > 0) {
> > +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!cam_dev->sensor) {
> > +		dev_err(dev, "Sensor is not connected\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	/* Seninf must stream on first */
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_dev_queue_req_buffers(cam_dev);
> > +	isp_composer_stream(isp_ctx, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		goto fail_sensor_off;
> > +	}
> > +
> > +	isp_composer_stream(isp_ctx, 0);
> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> 
> I'd be interested to get Tomasz's input here. If we fail to stream off
> one of the subdevs, should we stream on the other one? What if that
> fails? It's not clear to me the expectation when stream off fails, but
> this seems a bit odd.
> 

Ok, maybe we just return the error code to the user space in this case.
No need to perform stream on the other one.
If you have any better suggestion, please let us know.

> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> > +				   int enable)
> 
> This can fit on one line.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> > +					  struct v4l2_fh *fh,
> > +					  struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_link_setup(struct media_entity *entity,
> > +			      const struct media_pad *local,
> > +	const struct media_pad *remote, u32 flags)
> 
> Strange indentation here.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> > +		pad, remote->index);
> > +
> > +	if (pad < cam_dev->dev_node_num)
> > +		cam_dev->mem2mem2_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> > +		&buf->daddr, &buf->scp_addr);
> > +
> > +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct vb2_v4l2_buffer *v4l2_buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		v4l2_buf->request_fd,
> > +		v4l2_buf->vb2_buf.index);
> > +
> > +	if (v4l2_buf->request_fd < 0) {
> > +		mtk_cam_dev_queue_buffers(vb);
> > +		return;
> > +	}
> > +
> > +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> > +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> > +		mtk_cam_dev->request_count =
> > +			vb->req_obj.req->num_incomplete_objects;
> > +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> > +			v4l2_buf->request_fd,
> > +			vb->req_obj.req->num_incomplete_objects);
> > +	}
> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_cam_dev->request_count--;
> > +
> > +	if (!mtk_cam_dev->request_count) {
> > +		mtk_cam_dev->request_fd = -1;
> > +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> > +	}
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count > 0)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +	else
> > +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> > +
> > +	if (node->desc.smem_alloc) {
> > +		alloc_devs[0] = cam_dev->smem_dev;
> > +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> > +	} else {
> > +		alloc_devs[0] = &cam_dev->pdev->dev;
> > +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> > +			alloc_devs[0]);
> > +	}
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Validate initialized num_planes & size[0] */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	/* Initialize buffer queue & locks */
> > +	INIT_LIST_HEAD(&node->pending_list);
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> 
> Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will remove these initialization and only keep them in
mtk_cam_mem2mem2_v4l2_register function.

> > +
> > +	return 0;
> > +}
> > +
> > +static bool
> > +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> > +			    struct mtk_cam_video_device *except)
> > +{
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +
> > +		if (node == except)
> > +			continue;
> > +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> > +			return false;
> > +	}
> > +
> > +	return true;
> > +}
> > +
> > +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +				       struct mtk_cam_video_device *node,
> > +				       enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	unsigned int i;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	for (i = 0; i < node->vbq.num_buffers; ++i)
> > +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(node->vbq.bufs[i], state);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> > +			node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> > +			node->id, __func__);
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> > +		return 0;
> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> > +			node->id);
> > +		goto fail_stop_pipeline;
> > +	}
> > +	return 0;
> > +
> > +fail_stop_pipeline:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_return_bufs:
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +
> > +	/* Was this the first node with streaming disabled? */
> > +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> > +		/* Yes, really stop streaming now */
> > +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +			dev_err(&cam_dev->pdev->dev,
> > +				"failed to stop streaming\n");
> > +	}
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> > +			   struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> > +
> > +	if (in_fmt->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc,
> > +					     in_fmt);
> > +	}
> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> > +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> > +				       &in_fmt->fmt.pix_mp,
> > +				       node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_videoc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input)
> > +{
> > +	if (input->index > 0)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_CTRL:
> > +		return v4l2_ctrl_subscribe_event(fh, sub);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> > +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> > +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> > +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> > +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->index > 0 || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +/* subdev internal operations */
> > +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> > +	.open = mtk_cam_subdev_open,
> > +	.close = mtk_cam_subdev_close,
> > +};
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream = mtk_cam_subdev_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> > +	.link_setup = mtk_cam_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> > +}
> 
> Move this function up with the other mtk_cam_vb2_* functions.
> 

Fix in next patch.

> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +/*
> > + * Config node's video properties
> > + * according to the device context requirement
> > + */
> > +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> > +				 unsigned int node,
> > +				 struct video_device *vdev,
> > +				 struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev_node_desc *node_desc =
> > +		&cam_dev->mem2mem2_nodes[node].desc;
> > +
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> > +	f->type = node_desc->buf_type;
> > +	vdev->ioctl_ops = node_desc->ioctl_ops;
> > +
> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     node_desc,
> > +				     f);
> > +}
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = vb2_request_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	int ret;
> > +
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, media_dev);
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler)
> 
> This can be a static function.
> 

After reveiw, we will remove this function and move the source code to
the mtk_cam_mem2mem2_v4l2_register. Below error handling will be
removed, either.

> > +{
> > +	int ret;
> > +
> > +	/* Set up v4l2 device */
> > +	v4l2_dev->ctrl_handler = ctrl_handler;
> > +	v4l2_dev->mdev = media_dev;
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> > +	ret = v4l2_device_register(dev, v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> 
> The calling function will do this cleanup when this function fails, so
> no need to do it here; just return ret.
> 

Ditto.


> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int num_nodes = cam_dev->dev_node_num;
> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		goto fail_media_dev;
> > +	}
> > +
> > +	ret = mtk_cam_v4l2_register(dev,
> > +				    &cam_dev->media_dev,
> > +				    &cam_dev->v4l2_dev,
> > +				    NULL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> 
> It doesn't look like this gets free'd in any of the failure cases below.
> 

Based on Tomasz comments, we could use devm_kcalloc() here and remove
any matching calls to kfree(), since that would be cleaned up when
removing the driver.
https://patchwork.kernel.org/patch/10905223/
Do I have misunderstanding?

> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> > +
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < num_nodes; i++) {
> 
> Consider moving some of this loop to a new function. This would simplify
> the failure handling below by by removing the fail_vdev and
> fail_vdev_media_entity labels, whose cleanup could move into the helper
> function. It would also break up this large function a bit.
> 

Ok, thanks for your suggestion.
We will add new mtk_cam_video_register_device function to register video
device for each nodes and simplify the error handling.

	/* Create video nodes and links */
	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
		ret = mtk_cam_video_register_device(cam_dev, i);
		if (ret)
			goto fail_video_register;
	}

fail_video_register:
	i--;
	for (; i >= 0; i--) {
	video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
	media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
	mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
	}

> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +		struct video_device *vdev = &node->vdev;
> > +		struct vb2_queue *vbq = &node->vbq;
> > +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> > +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> > +
> > +		cam_dev->subdev_pads[i].flags = output ?
> > +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +		/* Initialize miscellaneous variables */
> > +		mutex_init(&node->lock);
> > +		spin_lock_init(&node->slock);
> > +		INIT_LIST_HEAD(&node->pending_list);
> > +
> > +		/* Initialize formats to default values */
> > +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> > +
> > +		/* Initialize media entities */
> > +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +		if (ret) {
> > +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +			goto fail_vdev_media_entity;
> > +		}
> > +		node->enabled = false;
> > +		node->id = i;
> > +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> > +		vdev->entity.ops = NULL;
> > +
> > +		/* Initialize vbq */
> > +		vbq->type = node->vdev_fmt.type;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +			vbq->io_modes = VB2_MMAP;
> > +		else
> > +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +		if (node->desc.smem_alloc)
> > +			vbq->bidirectional = 1;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +			vdev->entity.function =
> > +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +		vbq->ops = &mtk_cam_vb2_ops;
> > +		vbq->mem_ops = &vb2_dma_contig_memops;
> > +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +		/* Put the process hub sub device in the vb2 private data */
> > +		vbq->drv_priv = cam_dev;
> > +		vbq->lock = &node->lock;
> > +		vbq->supports_requests = true;
> > +
> > +		ret = vb2_queue_init(vbq);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Initialize vdev */
> > +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +		vdev->release = video_device_release_empty;
> > +		vdev->fops = &mtk_cam_v4l2_fops;
> > +		vdev->lock = &node->lock;
> > +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +		vdev->queue = &node->vbq;
> > +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +		/* Enable private control for image video devices */
> > +		if (node->desc.image) {
> > +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +			vdev->ctrl_handler = &node->ctrl_handler;
> > +		}
> > +		video_set_drvdata(vdev, cam_dev);
> > +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +		if (ret) {
> > +			dev_err(dev, "failed to register vde:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Create link between video node and the subdev pad */
> > +		if (output) {
> > +			ret = media_create_pad_link(&vdev->entity, 0,
> > +						    &cam_dev->subdev.entity,
> > +						    i, link_flags);
> > +		} else {
> > +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +						    i, &vdev->entity, 0,
> > +						    link_flags);
> > +		}
> > +		if (ret)
> > +			goto fail_link;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +	for (; i >= 0; i--) {
> > +fail_link:
> > +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> > +fail_vdev:
> > +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> > +fail_vdev_media_entity:
> > +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +fail_media_dev:
> >
> media_device_unregister and media_device_cleanup are called when
> mtk_cam_media_register fails, so only need to call these under the
> fail_v4l2_dev label.
> 

Will correct in next patch.

> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		dev = &cam_dev->mem2mem2_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_SENINF_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_P1_HUB_PAD_SINK,
> > +				    0);
> > +	if (ret)
> 
> should this function return an error here?
> 

Ok, we will print error log and just return here.

> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> 
> Should anything be done to cam_dev-seninf here, such as setting it to
> NULL?
> 

Ok, we will configure cam_dev-seninf to NULL here in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> > +				    struct v4l2_fwnode_endpoint *vep,
> > +				    struct v4l2_async_subdev *asd)
> > +{
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> > +		&cam_dev->pdev->dev, &cam_dev->notifier,
> > +		sizeof(struct v4l2_async_subdev),
> > +		mtk_cam_dev_fwnode_parse);
> 
> As far as I can tell, the fourth argument is optional, so I think you
> can just set NULL and remove mtk_cam_dev_fwnode_parse.
> 

Good point. Thanks for your suggestion.
Will fix in next patch.

> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > new file mode 100644
> > index 000000000000..73b36916da08
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > @@ -0,0 +1,43 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Frederic Chen <frederic.chen@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <media/v4l2-device.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap);
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes);
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_try_fmt(struct file *file,
> > +			   void *fh, struct v4l2_format *in_fmt);
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input);
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f);
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub);
> > +
> > +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> > -- 
> > 2.18.0
> > 

Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-28  1:00         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-28  1:00 UTC (permalink / raw)
  To: Drew Davenport
  Cc: tfiga, hans.verkuil, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi, Drew:

Appreciate your feedbacks on this patch set firstly.

On Fri, 2019-05-24 at 12:49 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and six video devices during
> > initialization. Moreover, it also connects with sensor and
> > senif drivers with V4L2 async APIs.
> 
> Thanks for the patch. I've made a few comments inline. As a general
> comment, what do you think of merging mtk_cam-dev.c and
> mtk_cam-v4l2-util.c into one file? They seem to call into one another
> and I'm not sure how beneficial it is to have them separate.
> 
> I have some comments on the other patches in this series that came about
> while I was reviewing this, which I will send as well.
> 
> [snip]
>  

Ok, we will merge tk_cam-dev.c into mtk_cam-v4l2-util.c in next patch
set.

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..5a581ab65945
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,19 @@
> > +#
> > +# Copyright (C) 2018 MediaTek Inc.
> > +#
> > +# This program is free software: you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License version 2 as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +
> > +mtk-cam-isp-objs += \
> > +	mtk_cam.o mtk_cam-dev.o \
> > +	mtk_cam-ctrl.o mtk_cam-scp.o \
> > +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
> 
> Some of these files are added in other patches. Consider adding files to
> the Makefile in the same patch a file is added.
> 

Ok, we will add the corresponding files in each patches.

> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > new file mode 100644
> > index 000000000000..dda8a7b161ee
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > @@ -0,0 +1,758 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + * Copyright (C) 2017 Google, Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/of.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-smem.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
> 
> This is not used anywhere. Please remove.
> 

Ok, we will remove unused variable.

> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> > +static struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct mtk_cam_dev_queues_setting queues_setting = {
> > +	.output_node_descs = output_queues,
> > +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> > +	.capture_node_descs = capture_queues,
> > +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> > +};
> 
> I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup
> 

Ok, we will remove mtk_cam_dev_queue_setup structure and revise
mtk_cam_dev_queue_setup function.

> > +
> > +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
> 
> Since only one_pixel_mode is supported, this function can be removed and
> the callsite replaced with ALIGN(size, 2). This function can be added
> once more when other pixel modes are supported.
> 

Got it, we will align_main_stream_size & align_packetd_out_size
functions and remove pix_mode argument in cal_main_stream_stride &
functions.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 4);
> > +	case one_pixel_mode:
> > +		return ALIGN(size, 2);
> > +	default:
> > +		break;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static unsigned int align_packetd_out_size(__u32 size,
> > +					   unsigned int pix_mode,
> > +					   __u32 fmt)
> 
> This is only ever called with one_pixel_mode. Remove the pix_mode
> argument and unreachable code.
> 

Ditto.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 16);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case one_pixel_mode:
> > +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> > +			return ALIGN(size, 4);
> > +		else
> > +			return ALIGN(size, 8);
> > +	default:
> > +		return ALIGN(size, 16);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static __u32 cal_main_stream_stride(struct device *dev,
> > +				    __u32 width,
> > +				    __u32 pix_fmt,
> > +				    __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_main_stream_size(stride, pix_mode);
> > +
> > +	dev_dbg(dev,
> > +		"main width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_packed_out_stride(struct device *dev,
> > +				   __u32 width,
> > +				   __u32 pix_fmt,
> > +				   __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = DIV_ROUND_UP(width * 3, 2);
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> > +
> > +	dev_dbg(dev,
> > +		"packed width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_img_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> > +					     one_pixel_mode);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> > +					    one_pixel_mode);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* The helper to configure the device context */
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting)
> 
> This is only ever called with the same mtk_cam_dev_queues_setting
> struct. I think you can remove that struct altogether and just set the
> mtk_cam_dev_node_desc* for each node from output_queues and
> capture_queues directly.
> 
> Also this can be a static function.
> 

Thanks for your suggestion.
We will revise in next patch as below.

/* The helper to configure the device context */
static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)

> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < setting->total_output_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->output_node_descs[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < setting->total_capture_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->capture_node_descs[i];
> > +
> > +	cam_dev->dev_node_num = node_idx;
> 
> This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
> MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
> that instead of dev_node_num?
> 

Ok, we will use new MTK_CAM_P1_TOTAL_NODES const value to replace
dev_node_num in next patch.

> > +}
> > +
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *fram_param)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +
> > +	if (!cam_dev->streaming)
> > +		return 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev,
> > +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> > +		fram_param->request_fd,
> > +		fram_param->frame_seq_no,
> > +		fram_param->state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> > +		buf->vbb.sequence = fram_param->frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +
> > +	return 0;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = cal_img_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> > +}
> > +
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> > +}
> > +
> > +/* Get the default format setting */
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue_desc,
> > +				  struct v4l2_format *dest)
> > +{
> > +	struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_fmt_set_img(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +/* Get a free buffer from a video node */
> > +static struct mtk_cam_dev_buffer *
> > +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> > +{
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct mtk_cam_video_device *vdev;
> > +
> > +	if (node > cam_dev->dev_node_num || node < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> > +		return NULL;
> > +	}
> > +	vdev = &cam_dev->mem2mem2_nodes[node];
> > +
> > +	spin_lock(&vdev->slock);
> > +	buf = list_first_entry_or_null(&vdev->pending_list,
> > +				       struct mtk_cam_dev_buffer,
> > +				       list);
> > +	if (!buf) {
> > +		spin_unlock(&vdev->slock);
> > +		return NULL;
> > +	}
> > +	list_del(&buf->list);
> > +	spin_unlock(&vdev->slock);
> 
> Can this be simplified by going:
> spin_lock();
> buf = list_first_entry_or_null(...);
> if (buf) list_del(...);
> spin_unlock();
> return buf;
> 

Ok, we will simplify this implementation as you suggested.

> > +
> > +	return buf;
> > +}
> > +
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
> 
> This only ever returns 0, so make it a void function.
> 

Ok, fix in next patch.

> > +{
> > +	unsigned int node;
> > +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_start_param s_param;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> > +		return 0;
> > +	}
> > +
> > +	/* Check all enabled nodes to collect its buffer  */
> > +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> > +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> > +			continue;
> > +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> > +		if (!buf)
> > +			continue;
> > +
> > +		/* TBD: use buf_init callback function */
> > +		buf->daddr =
> > +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> > +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> > +				cam_dev->smem_dev, buf->daddr);
> > +		} else {
> > +			buf->scp_addr = 0;
> > +		}
> > +
> > +		dev_dbg(dev,
> > +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> > +			node,
> > +			buf->vbb.request_fd,
> > +			buf->vbb.vb2_buf.index,
> > +			buf->vbb.vb2_buf.state,
> > +			&buf->daddr,
> > +			&buf->scp_addr);
> > +
> > +		s_param.buffers[node] = buf;
> > +		s_param.request_fd = buf->vbb.request_fd;
> > +	}
> > +
> > +	/* Trigger en-queued job to driver */
> > +	mtk_isp_req_enqueue(dev, &s_param);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> > +
> > +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> > +
> > +	/* v4l2 sub-device registration */
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> > +
> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret)
> 
> If this fails do we need to undo the stuff done in
> mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will call mtk_cam_v4l2_unregister(cam_dev) if failed in
mtk_cam_v4l2_async_register function in next patch.


> > +		return ret;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > new file mode 100644
> > index 000000000000..410460de44fa
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > @@ -0,0 +1,250 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_H__
> > +#define __MTK_CAM_DEV_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> > +
> > +#define MTK_CAM_DEV_NODES		11
> > +
> > +#define MTK_CAM_P1_META_IN_0		0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> > +#define MTK_CAM_P1_META_OUT_0		3
> > +#define MTK_CAM_P1_META_OUT_1		4
> > +#define MTK_CAM_P1_META_OUT_2		5
> > +#define MTK_CAM_P1_META_OUT_3		6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE	6
> 
> Please align macro values using tabs.
> 

Ok, we will fix this coding style issue in next patch.

> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +};
> > +
> > +/* Attributes setup by device owner */
> > +struct mtk_cam_dev_queues_setting {
> > +	const struct mtk_cam_dev_node_desc *output_node_descs;
> > +	unsigned int total_output_nodes;
> > +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> > +	unsigned int total_capture_nodes;
> > +};
> > +
> > +struct mtk_cam_dev_start_param {
> > +	int request_fd;
> > +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> > +};
> > +
> > +struct mtk_cam_dev_finish_param {
> > +	int request_fd;
> > +	unsigned int frame_seq_no;
> > +	unsigned int state;
> > +	struct list_head *list_buf;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	struct v4l2_format *fmts;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct video_device vdev;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> > +	struct mutex lock;
> > +	/* protect for pending_list */
> > +	spinlock_t slock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @dev_node_num: The number of supported V4L2 video device nodes
> > + * @request_fd:	The file descriptor of request API
> > + * @request_count: The buffer count of request API
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int dev_node_num;
> > +	int request_fd;
> > +	unsigned int request_count;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler);
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting);
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *param);
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no);
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id);
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue,
> > +				  struct v4l2_format *dest_fmt);
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id);
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_dev *
> > +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> > +{
> > +	return container_of(__sd,
> > +		struct mtk_cam_dev, subdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> > +{
> > +	return container_of(__vq,
> > +		struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb,
> > +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +#endif /* __MTK_CAM_DEV_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > new file mode 100644
> > index 000000000000..196aaef3d854
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > @@ -0,0 +1,1086 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define MTK_CAM_SENINF_PAD_SRC			4
> > +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> > +
> > +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> > +			       struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	cam_dev->request_fd = -1;
> > +	cam_dev->request_count = 0;
> > +
> > +	return mtk_isp_open(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> > +				struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	return mtk_isp_release(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
> 
> "get" implies that this function will retrieve something without
> side effects, which is not the case here. In the error case, the return
> value is ignored by the caller as well.
> 
> Consider making this function return a struct v4l2_subdev* (or NULL in
> the error case) and let the caller set mtk_cam_dev::sensor.
> 

Ok, below is the new function prototype.
static struct v4l2_subdev *
mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)

> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	cam_dev->sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count > 0) {
> > +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!cam_dev->sensor) {
> > +		dev_err(dev, "Sensor is not connected\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	/* Seninf must stream on first */
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_dev_queue_req_buffers(cam_dev);
> > +	isp_composer_stream(isp_ctx, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		goto fail_sensor_off;
> > +	}
> > +
> > +	isp_composer_stream(isp_ctx, 0);
> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> 
> I'd be interested to get Tomasz's input here. If we fail to stream off
> one of the subdevs, should we stream on the other one? What if that
> fails? It's not clear to me the expectation when stream off fails, but
> this seems a bit odd.
> 

Ok, maybe we just return the error code to the user space in this case.
No need to perform stream on the other one.
If you have any better suggestion, please let us know.

> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> > +				   int enable)
> 
> This can fit on one line.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> > +					  struct v4l2_fh *fh,
> > +					  struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_link_setup(struct media_entity *entity,
> > +			      const struct media_pad *local,
> > +	const struct media_pad *remote, u32 flags)
> 
> Strange indentation here.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> > +		pad, remote->index);
> > +
> > +	if (pad < cam_dev->dev_node_num)
> > +		cam_dev->mem2mem2_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> > +		&buf->daddr, &buf->scp_addr);
> > +
> > +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct vb2_v4l2_buffer *v4l2_buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		v4l2_buf->request_fd,
> > +		v4l2_buf->vb2_buf.index);
> > +
> > +	if (v4l2_buf->request_fd < 0) {
> > +		mtk_cam_dev_queue_buffers(vb);
> > +		return;
> > +	}
> > +
> > +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> > +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> > +		mtk_cam_dev->request_count =
> > +			vb->req_obj.req->num_incomplete_objects;
> > +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> > +			v4l2_buf->request_fd,
> > +			vb->req_obj.req->num_incomplete_objects);
> > +	}
> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_cam_dev->request_count--;
> > +
> > +	if (!mtk_cam_dev->request_count) {
> > +		mtk_cam_dev->request_fd = -1;
> > +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> > +	}
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count > 0)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +	else
> > +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> > +
> > +	if (node->desc.smem_alloc) {
> > +		alloc_devs[0] = cam_dev->smem_dev;
> > +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> > +	} else {
> > +		alloc_devs[0] = &cam_dev->pdev->dev;
> > +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> > +			alloc_devs[0]);
> > +	}
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Validate initialized num_planes & size[0] */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	/* Initialize buffer queue & locks */
> > +	INIT_LIST_HEAD(&node->pending_list);
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> 
> Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will remove these initialization and only keep them in
mtk_cam_mem2mem2_v4l2_register function.

> > +
> > +	return 0;
> > +}
> > +
> > +static bool
> > +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> > +			    struct mtk_cam_video_device *except)
> > +{
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +
> > +		if (node == except)
> > +			continue;
> > +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> > +			return false;
> > +	}
> > +
> > +	return true;
> > +}
> > +
> > +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +				       struct mtk_cam_video_device *node,
> > +				       enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	unsigned int i;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	for (i = 0; i < node->vbq.num_buffers; ++i)
> > +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(node->vbq.bufs[i], state);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> > +			node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> > +			node->id, __func__);
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> > +		return 0;
> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> > +			node->id);
> > +		goto fail_stop_pipeline;
> > +	}
> > +	return 0;
> > +
> > +fail_stop_pipeline:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_return_bufs:
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +
> > +	/* Was this the first node with streaming disabled? */
> > +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> > +		/* Yes, really stop streaming now */
> > +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +			dev_err(&cam_dev->pdev->dev,
> > +				"failed to stop streaming\n");
> > +	}
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> > +			   struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> > +
> > +	if (in_fmt->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc,
> > +					     in_fmt);
> > +	}
> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> > +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> > +				       &in_fmt->fmt.pix_mp,
> > +				       node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_videoc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input)
> > +{
> > +	if (input->index > 0)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_CTRL:
> > +		return v4l2_ctrl_subscribe_event(fh, sub);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> > +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> > +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> > +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> > +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->index > 0 || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +/* subdev internal operations */
> > +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> > +	.open = mtk_cam_subdev_open,
> > +	.close = mtk_cam_subdev_close,
> > +};
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream = mtk_cam_subdev_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> > +	.link_setup = mtk_cam_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> > +}
> 
> Move this function up with the other mtk_cam_vb2_* functions.
> 

Fix in next patch.

> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +/*
> > + * Config node's video properties
> > + * according to the device context requirement
> > + */
> > +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> > +				 unsigned int node,
> > +				 struct video_device *vdev,
> > +				 struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev_node_desc *node_desc =
> > +		&cam_dev->mem2mem2_nodes[node].desc;
> > +
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> > +	f->type = node_desc->buf_type;
> > +	vdev->ioctl_ops = node_desc->ioctl_ops;
> > +
> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     node_desc,
> > +				     f);
> > +}
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = vb2_request_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	int ret;
> > +
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, media_dev);
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler)
> 
> This can be a static function.
> 

After reveiw, we will remove this function and move the source code to
the mtk_cam_mem2mem2_v4l2_register. Below error handling will be
removed, either.

> > +{
> > +	int ret;
> > +
> > +	/* Set up v4l2 device */
> > +	v4l2_dev->ctrl_handler = ctrl_handler;
> > +	v4l2_dev->mdev = media_dev;
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> > +	ret = v4l2_device_register(dev, v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> 
> The calling function will do this cleanup when this function fails, so
> no need to do it here; just return ret.
> 

Ditto.


> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int num_nodes = cam_dev->dev_node_num;
> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		goto fail_media_dev;
> > +	}
> > +
> > +	ret = mtk_cam_v4l2_register(dev,
> > +				    &cam_dev->media_dev,
> > +				    &cam_dev->v4l2_dev,
> > +				    NULL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> 
> It doesn't look like this gets free'd in any of the failure cases below.
> 

Based on Tomasz comments, we could use devm_kcalloc() here and remove
any matching calls to kfree(), since that would be cleaned up when
removing the driver.
https://patchwork.kernel.org/patch/10905223/
Do I have misunderstanding?

> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> > +
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < num_nodes; i++) {
> 
> Consider moving some of this loop to a new function. This would simplify
> the failure handling below by by removing the fail_vdev and
> fail_vdev_media_entity labels, whose cleanup could move into the helper
> function. It would also break up this large function a bit.
> 

Ok, thanks for your suggestion.
We will add new mtk_cam_video_register_device function to register video
device for each nodes and simplify the error handling.

	/* Create video nodes and links */
	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
		ret = mtk_cam_video_register_device(cam_dev, i);
		if (ret)
			goto fail_video_register;
	}

fail_video_register:
	i--;
	for (; i >= 0; i--) {
	video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
	media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
	mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
	}

> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +		struct video_device *vdev = &node->vdev;
> > +		struct vb2_queue *vbq = &node->vbq;
> > +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> > +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> > +
> > +		cam_dev->subdev_pads[i].flags = output ?
> > +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +		/* Initialize miscellaneous variables */
> > +		mutex_init(&node->lock);
> > +		spin_lock_init(&node->slock);
> > +		INIT_LIST_HEAD(&node->pending_list);
> > +
> > +		/* Initialize formats to default values */
> > +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> > +
> > +		/* Initialize media entities */
> > +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +		if (ret) {
> > +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +			goto fail_vdev_media_entity;
> > +		}
> > +		node->enabled = false;
> > +		node->id = i;
> > +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> > +		vdev->entity.ops = NULL;
> > +
> > +		/* Initialize vbq */
> > +		vbq->type = node->vdev_fmt.type;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +			vbq->io_modes = VB2_MMAP;
> > +		else
> > +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +		if (node->desc.smem_alloc)
> > +			vbq->bidirectional = 1;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +			vdev->entity.function =
> > +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +		vbq->ops = &mtk_cam_vb2_ops;
> > +		vbq->mem_ops = &vb2_dma_contig_memops;
> > +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +		/* Put the process hub sub device in the vb2 private data */
> > +		vbq->drv_priv = cam_dev;
> > +		vbq->lock = &node->lock;
> > +		vbq->supports_requests = true;
> > +
> > +		ret = vb2_queue_init(vbq);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Initialize vdev */
> > +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +		vdev->release = video_device_release_empty;
> > +		vdev->fops = &mtk_cam_v4l2_fops;
> > +		vdev->lock = &node->lock;
> > +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +		vdev->queue = &node->vbq;
> > +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +		/* Enable private control for image video devices */
> > +		if (node->desc.image) {
> > +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +			vdev->ctrl_handler = &node->ctrl_handler;
> > +		}
> > +		video_set_drvdata(vdev, cam_dev);
> > +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +		if (ret) {
> > +			dev_err(dev, "failed to register vde:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Create link between video node and the subdev pad */
> > +		if (output) {
> > +			ret = media_create_pad_link(&vdev->entity, 0,
> > +						    &cam_dev->subdev.entity,
> > +						    i, link_flags);
> > +		} else {
> > +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +						    i, &vdev->entity, 0,
> > +						    link_flags);
> > +		}
> > +		if (ret)
> > +			goto fail_link;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +	for (; i >= 0; i--) {
> > +fail_link:
> > +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> > +fail_vdev:
> > +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> > +fail_vdev_media_entity:
> > +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +fail_media_dev:
> >
> media_device_unregister and media_device_cleanup are called when
> mtk_cam_media_register fails, so only need to call these under the
> fail_v4l2_dev label.
> 

Will correct in next patch.

> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		dev = &cam_dev->mem2mem2_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_SENINF_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_P1_HUB_PAD_SINK,
> > +				    0);
> > +	if (ret)
> 
> should this function return an error here?
> 

Ok, we will print error log and just return here.

> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> 
> Should anything be done to cam_dev-seninf here, such as setting it to
> NULL?
> 

Ok, we will configure cam_dev-seninf to NULL here in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> > +				    struct v4l2_fwnode_endpoint *vep,
> > +				    struct v4l2_async_subdev *asd)
> > +{
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> > +		&cam_dev->pdev->dev, &cam_dev->notifier,
> > +		sizeof(struct v4l2_async_subdev),
> > +		mtk_cam_dev_fwnode_parse);
> 
> As far as I can tell, the fourth argument is optional, so I think you
> can just set NULL and remove mtk_cam_dev_fwnode_parse.
> 

Good point. Thanks for your suggestion.
Will fix in next patch.

> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > new file mode 100644
> > index 000000000000..73b36916da08
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > @@ -0,0 +1,43 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Frederic Chen <frederic.chen@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <media/v4l2-device.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap);
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes);
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_try_fmt(struct file *file,
> > +			   void *fh, struct v4l2_format *in_fmt);
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input);
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f);
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub);
> > +
> > +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> > -- 
> > 2.18.0
> > 

Best regards,


Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-05-28  1:00         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-05-28  1:00 UTC (permalink / raw)
  To: Drew Davenport
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, hans.verkuil, frederic.chen,
	seraph.huang, linux-media, devicetree, shik, yuzhao,
	linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, christie.yu,
	zwisler

Hi, Drew:

Appreciate your feedbacks on this patch set firstly.

On Fri, 2019-05-24 at 12:49 -0600, Drew Davenport wrote:
> Hi Jungo,
> 
> On Fri, May 10, 2019 at 09:58:02AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and six video devices during
> > initialization. Moreover, it also connects with sensor and
> > senif drivers with V4L2 async APIs.
> 
> Thanks for the patch. I've made a few comments inline. As a general
> comment, what do you think of merging mtk_cam-dev.c and
> mtk_cam-v4l2-util.c into one file? They seem to call into one another
> and I'm not sure how beneficial it is to have them separate.
> 
> I have some comments on the other patches in this series that came about
> while I was reviewing this, which I will send as well.
> 
> [snip]
>  

Ok, we will merge tk_cam-dev.c into mtk_cam-v4l2-util.c in next patch
set.

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..5a581ab65945
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,19 @@
> > +#
> > +# Copyright (C) 2018 MediaTek Inc.
> > +#
> > +# This program is free software: you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License version 2 as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +
> > +mtk-cam-isp-objs += \
> > +	mtk_cam.o mtk_cam-dev.o \
> > +	mtk_cam-ctrl.o mtk_cam-scp.o \
> > +	mtk_cam-v4l2-util.o mtk_cam-smem-dev.o
> 
> Some of these files are added in other patches. Consider adding files to
> the Makefile in the same patch a file is added.
> 

Ok, we will add the corresponding files in each patches.

> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1_SUPPORT) += mtk-cam-isp.o
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > new file mode 100644
> > index 000000000000..dda8a7b161ee
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.c
> > @@ -0,0 +1,758 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + * Copyright (C) 2017 Google, Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/of.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-smem.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = mtk_cam_vidioc_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vout_ioctl_ops = {
> 
> This is not used anywhere. Please remove.
> 

Ok, we will remove unused variable.

> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_out_mplane = mtk_cam_videoc_enum_fmt,
> > +	.vidioc_g_fmt_vid_out_mplane = mtk_cam_videoc_g_fmt,
> > +	.vidioc_s_fmt_vid_out_mplane = mtk_cam_videoc_s_fmt,
> > +	.vidioc_try_fmt_vid_out_mplane = mtk_cam_videoc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> > +	/* buffer queue management */
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_videoc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_meta_enum_format,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_videoc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +/* Need to update mtk_cam_dev_fmt_set_img for default format configuration */
> > +static struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = MEDIA_LNK_FL_DYNAMIC,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct mtk_cam_dev_queues_setting queues_setting = {
> > +	.output_node_descs = output_queues,
> > +	.total_output_nodes = MTK_CAM_P1_TOTAL_OUTPUT,
> > +	.capture_node_descs = capture_queues,
> > +	.total_capture_nodes = MTK_CAM_P1_TOTAL_CAPTURE,
> > +};
> 
> I think this struct can be removed. See my comment in mtk_cam_dev_queue_setup
> 

Ok, we will remove mtk_cam_dev_queue_setup structure and revise
mtk_cam_dev_queue_setup function.

> > +
> > +static __u32 get_pixel_byte_by_fmt(__u32 pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	case V4L2_PIX_FMT_MTISP_U8:
> > +	case V4L2_PIX_FMT_MTISP_U10:
> > +	case V4L2_PIX_FMT_MTISP_U12:
> > +	case V4L2_PIX_FMT_MTISP_U14:
> > +		return 16;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 align_main_stream_size(__u32 size, unsigned int pix_mode)
> 
> Since only one_pixel_mode is supported, this function can be removed and
> the callsite replaced with ALIGN(size, 2). This function can be added
> once more when other pixel modes are supported.
> 

Got it, we will align_main_stream_size & align_packetd_out_size
functions and remove pix_mode argument in cal_main_stream_stride &
functions.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 4);
> > +	case one_pixel_mode:
> > +		return ALIGN(size, 2);
> > +	default:
> > +		break;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static unsigned int align_packetd_out_size(__u32 size,
> > +					   unsigned int pix_mode,
> > +					   __u32 fmt)
> 
> This is only ever called with one_pixel_mode. Remove the pix_mode
> argument and unreachable code.
> 

Ditto.

> > +{
> > +	switch (pix_mode) {
> > +	case default_pixel_mode:
> > +	case four_pixel_mode:
> > +		return ALIGN(size, 16);
> > +	case two_pixel_mode:
> > +		return ALIGN(size, 8);
> > +	case one_pixel_mode:
> > +		if (fmt == V4L2_PIX_FMT_MTISP_F10)
> > +			return ALIGN(size, 4);
> > +		else
> > +			return ALIGN(size, 8);
> > +	default:
> > +		return ALIGN(size, 16);
> > +	}
> > +	return 0;
> > +}
> > +
> > +static __u32 cal_main_stream_stride(struct device *dev,
> > +				    __u32 width,
> > +				    __u32 pix_fmt,
> > +				    __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_main_stream_size(stride, pix_mode);
> > +
> > +	dev_dbg(dev,
> > +		"main width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_packed_out_stride(struct device *dev,
> > +				   __u32 width,
> > +				   __u32 pix_fmt,
> > +				   __u32 pix_mode)
> 
> This function is only called with one_pixel_mode. Remove the pix_mode
> argument.
> 

Ditto.

> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> > +	stride = DIV_ROUND_UP(width * 3, 2);
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +	/* expand stride, instead of shrink width */
> > +	stride = align_packetd_out_size(stride, pix_mode, pix_fmt);
> > +
> > +	dev_dbg(dev,
> > +		"packed width:%d, pix_mode:%d, stride:%d\n",
> > +		width, pix_mode, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 cal_img_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = cal_main_stream_stride(dev, width, pix_fmt,
> > +					     one_pixel_mode);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = cal_packed_out_stride(dev, width, pix_fmt,
> > +					    one_pixel_mode);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* The helper to configure the device context */
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting)
> 
> This is only ever called with the same mtk_cam_dev_queues_setting
> struct. I think you can remove that struct altogether and just set the
> mtk_cam_dev_node_desc* for each node from output_queues and
> capture_queues directly.
> 
> Also this can be a static function.
> 

Thanks for your suggestion.
We will revise in next patch as below.

/* The helper to configure the device context */
static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)

> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < setting->total_output_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->output_node_descs[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < setting->total_capture_nodes; i++)
> > +		cam_dev->mem2mem2_nodes[node_idx++].desc =
> > +			setting->capture_node_descs[i];
> > +
> > +	cam_dev->dev_node_num = node_idx;
> 
> This value is known at compile time (MTK_CAM_P1_TOTAL_OUTPUT +
> MTK_CAM_P1_TOTAL_CAPTURE). Can we just #define that constant and use
> that instead of dev_node_num?
> 

Ok, we will use new MTK_CAM_P1_TOTAL_NODES const value to replace
dev_node_num in next patch.

> > +}
> > +
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *fram_param)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +
> > +	if (!cam_dev->streaming)
> > +		return 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev,
> > +		"job recvied request fd:%d, frame_seq:%d state:%d\n",
> > +		fram_param->request_fd,
> > +		fram_param->frame_seq_no,
> > +		fram_param->state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	list_for_each_entry_safe(buf, b0, fram_param->list_buf, list) {
> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = ktime_get_ns();
> > +		buf->vbb.sequence = fram_param->frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(&buf->vbb.vb2_buf, fram_param->state);
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +
> > +	return 0;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = cal_img_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> > +}
> > +
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_pix_fmt(dev, dest_fmt, node_id);
> > +}
> > +
> > +/* Get the default format setting */
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue_desc,
> > +				  struct v4l2_format *dest)
> > +{
> > +	struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_fmt_set_img(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +/* Get a free buffer from a video node */
> > +static struct mtk_cam_dev_buffer *
> > +mtk_cam_dev_get_pending_buf(struct mtk_cam_dev *cam_dev, int node)
> > +{
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct mtk_cam_video_device *vdev;
> > +
> > +	if (node > cam_dev->dev_node_num || node < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Invalid mtk_cam_dev node.\n");
> > +		return NULL;
> > +	}
> > +	vdev = &cam_dev->mem2mem2_nodes[node];
> > +
> > +	spin_lock(&vdev->slock);
> > +	buf = list_first_entry_or_null(&vdev->pending_list,
> > +				       struct mtk_cam_dev_buffer,
> > +				       list);
> > +	if (!buf) {
> > +		spin_unlock(&vdev->slock);
> > +		return NULL;
> > +	}
> > +	list_del(&buf->list);
> > +	spin_unlock(&vdev->slock);
> 
> Can this be simplified by going:
> spin_lock();
> buf = list_first_entry_or_null(...);
> if (buf) list_del(...);
> spin_unlock();
> return buf;
> 

Ok, we will simplify this implementation as you suggested.

> > +
> > +	return buf;
> > +}
> > +
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev)
> 
> This only ever returns 0, so make it a void function.
> 

Ok, fix in next patch.

> > +{
> > +	unsigned int node;
> > +	const int mtk_cam_dev_node_num = cam_dev->dev_node_num;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_start_param s_param;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	memset(&s_param, 0, sizeof(struct mtk_cam_dev_start_param));
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_dbg(dev, "%s: stream off, no enqueue\n", __func__);
> > +		return 0;
> > +	}
> > +
> > +	/* Check all enabled nodes to collect its buffer  */
> > +	for (node = 0; node < mtk_cam_dev_node_num; node++) {
> > +		if (!cam_dev->mem2mem2_nodes[node].enabled)
> > +			continue;
> > +		buf = mtk_cam_dev_get_pending_buf(cam_dev, node);
> > +		if (!buf)
> > +			continue;
> > +
> > +		/* TBD: use buf_init callback function */
> > +		buf->daddr =
> > +			vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +		if (cam_dev->mem2mem2_nodes[node].desc.smem_alloc) {
> > +			buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(
> > +				cam_dev->smem_dev, buf->daddr);
> > +		} else {
> > +			buf->scp_addr = 0;
> > +		}
> > +
> > +		dev_dbg(dev,
> > +			"Node:%d fd:%d idx:%d state:%d daddr:%pad addr:%pad",
> > +			node,
> > +			buf->vbb.request_fd,
> > +			buf->vbb.vb2_buf.index,
> > +			buf->vbb.vb2_buf.state,
> > +			&buf->daddr,
> > +			&buf->scp_addr);
> > +
> > +		s_param.buffers[node] = buf;
> > +		s_param.request_fd = buf->vbb.request_fd;
> > +	}
> > +
> > +	/* Trigger en-queued job to driver */
> > +	mtk_isp_req_enqueue(dev, &s_param);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> > +
> > +	mtk_cam_dev_queue_setup(cam_dev, &queues_setting);
> > +
> > +	/* v4l2 sub-device registration */
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> > +
> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret)
> 
> If this fails do we need to undo the stuff done in
> mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will call mtk_cam_v4l2_unregister(cam_dev) if failed in
mtk_cam_v4l2_async_register function in next patch.


> > +		return ret;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > new file mode 100644
> > index 000000000000..410460de44fa
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-dev.h
> > @@ -0,0 +1,250 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-dev is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_H__
> > +#define __MTK_CAM_DEV_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME		"MTK-ISP-P1-V4L2"
> > +
> > +#define MTK_CAM_DEV_NODES		11
> > +
> > +#define MTK_CAM_P1_META_IN_0		0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT	1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT	2
> > +#define MTK_CAM_P1_META_OUT_0		3
> > +#define MTK_CAM_P1_META_OUT_1		4
> > +#define MTK_CAM_P1_META_OUT_2		5
> > +#define MTK_CAM_P1_META_OUT_3		6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE	6
> 
> Please align macro values using tabs.
> 

Ok, we will fix this coding style issue in next patch.

> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +};
> > +
> > +/* Attributes setup by device owner */
> > +struct mtk_cam_dev_queues_setting {
> > +	const struct mtk_cam_dev_node_desc *output_node_descs;
> > +	unsigned int total_output_nodes;
> > +	const struct mtk_cam_dev_node_desc *capture_node_descs;
> > +	unsigned int total_capture_nodes;
> > +};
> > +
> > +struct mtk_cam_dev_start_param {
> > +	int request_fd;
> > +	struct mtk_cam_dev_buffer *buffers[MTK_CAM_DEV_NODES];
> > +};
> > +
> > +struct mtk_cam_dev_finish_param {
> > +	int request_fd;
> > +	unsigned int frame_seq_no;
> > +	unsigned int state;
> > +	struct list_head *list_buf;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	struct v4l2_format *fmts;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct video_device vdev;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> > +	struct mutex lock;
> > +	/* protect for pending_list */
> > +	spinlock_t slock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @mem2mem2_nodes: The array list of mtk_cam_video_device
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @dev_node_num: The number of supported V4L2 video device nodes
> > + * @request_fd:	The file descriptor of request API
> > + * @request_count: The buffer count of request API
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device mem2mem2_nodes[MTK_CAM_DEV_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int dev_node_num;
> > +	int request_fd;
> > +	unsigned int request_count;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler);
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_queue_req_buffers(struct mtk_cam_dev *cam_dev);
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> > +			struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev,
> > +			     const struct mtk_cam_dev_queues_setting *setting);
> > +int mtk_cam_dev_job_finish(struct mtk_cam_dev *cam_dev,
> > +			   struct mtk_cam_dev_finish_param *param);
> > +int mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				 __u32 frame_seq_no);
> > +void mtk_cam_dev_fmt_set_img(struct device *dev,
> > +			     struct v4l2_pix_format_mplane *dest_fmt,
> > +			     struct v4l2_pix_format_mplane *src_fmt,
> > +			     unsigned int node_id);
> > +struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *queue_desc, u32 format);
> > +void mtk_cam_dev_load_default_fmt(struct device *dev,
> > +				  struct mtk_cam_dev_node_desc *queue,
> > +				  struct v4l2_format *dest_fmt);
> > +void mtk_cam_dev_cal_mplane_pix_fmt(struct device *dev,
> > +				    struct v4l2_pix_format_mplane *dest_fmt,
> > +				    unsigned int node_id);
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_dev *
> > +mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
> > +{
> > +	return container_of(__sd,
> > +		struct mtk_cam_dev, subdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> > +{
> > +	return container_of(__vq,
> > +		struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb,
> > +		struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +#endif /* __MTK_CAM_DEV_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > new file mode 100644
> > index 000000000000..196aaef3d854
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> > @@ -0,0 +1,1086 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Mediatek Corporation.
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * MTK_CAM-v4l2 is highly based on Intel IPU3 ImgU driver.
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <linux/device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ctrl.h"
> > +#include "mtk_cam-dev.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define MTK_CAM_SENINF_PAD_SRC			4
> > +#define MTK_CAM_P1_HUB_PAD_SINK			MTK_CAM_DEV_NODES
> > +
> > +static int mtk_cam_subdev_open(struct v4l2_subdev *sd,
> > +			       struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	cam_dev->request_fd = -1;
> > +	cam_dev->request_count = 0;
> > +
> > +	return mtk_isp_open(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_subdev_close(struct v4l2_subdev *sd,
> > +				struct v4l2_subdev_fh *fh)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	return mtk_isp_release(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_v4l2_get_active_sensor(struct mtk_cam_dev *cam_dev)
> 
> "get" implies that this function will retrieve something without
> side effects, which is not the case here. In the error case, the return
> value is ignored by the caller as well.
> 
> Consider making this function return a struct v4l2_subdev* (or NULL in
> the error case) and let the caller set mtk_cam_dev::sensor.
> 

Ok, below is the new function prototype.
static struct v4l2_subdev *
mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)

> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	cam_dev->sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count > 0) {
> > +			cam_dev->sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!cam_dev->sensor) {
> > +		dev_err(dev, "Sensor is not connected\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	ret = mtk_cam_v4l2_get_active_sensor(cam_dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> > +
> > +	/* Seninf must stream on first */
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "streamed on: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_dev_queue_req_buffers(cam_dev);
> > +	isp_composer_stream(isp_ctx, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->sensor->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	dev_dbg(dev, "stream off: %s\n", cam_dev->seninf->entity.name);
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		goto fail_sensor_off;
> > +	}
> > +
> > +	isp_composer_stream(isp_ctx, 0);
> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> 
> I'd be interested to get Tomasz's input here. If we fail to stream off
> one of the subdevs, should we stream on the other one? What if that
> fails? It's not clear to me the expectation when stream off fails, but
> this seems a bit odd.
> 

Ok, maybe we just return the error code to the user space in this case.
No need to perform stream on the other one.
If you have any better suggestion, please let us know.

> > +	return -EPERM;
> > +}
> > +
> > +static int mtk_cam_subdev_s_stream(struct v4l2_subdev *sd,
> > +				   int enable)
> 
> This can fit on one line.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_subdev_subscribe_event(struct v4l2_subdev *subdev,
> > +					  struct v4l2_fh *fh,
> > +					  struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_link_setup(struct media_entity *entity,
> > +			      const struct media_pad *local,
> > +	const struct media_pad *remote, u32 flags)
> 
> Strange indentation here.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "link setup: %d -> %d\n",
> > +		pad, remote->index);
> > +
> > +	if (pad < cam_dev->dev_node_num)
> > +		cam_dev->mem2mem2_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_queue_buffers(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%pad:%pad\n",
> > +		&buf->daddr, &buf->scp_addr);
> > +
> > +	mtk_isp_enqueue(&cam_dev->pdev->dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct vb2_v4l2_buffer *v4l2_buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		v4l2_buf->request_fd,
> > +		v4l2_buf->vb2_buf.index);
> > +
> > +	if (v4l2_buf->request_fd < 0) {
> > +		mtk_cam_dev_queue_buffers(vb);
> > +		return;
> > +	}
> > +
> > +	if (mtk_cam_dev->request_fd != v4l2_buf->request_fd) {
> > +		mtk_cam_dev->request_fd = v4l2_buf->request_fd;
> > +		mtk_cam_dev->request_count =
> > +			vb->req_obj.req->num_incomplete_objects;
> > +		dev_dbg(dev, "init  mtk_cam_dev_buf, fd(%d) count(%d)\n",
> > +			v4l2_buf->request_fd,
> > +			vb->req_obj.req->num_incomplete_objects);
> > +	}
> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_cam_dev->request_count--;
> > +
> > +	if (!mtk_cam_dev->request_count) {
> > +		mtk_cam_dev->request_fd = -1;
> > +		mtk_cam_dev_queue_req_buffers(mtk_cam_dev);
> > +	}
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count > 0)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +	else
> > +		*num_buffers = clamp_val(*num_buffers, 1, VB2_MAX_FRAME);
> > +
> > +	if (node->desc.smem_alloc) {
> > +		alloc_devs[0] = cam_dev->smem_dev;
> > +		dev_dbg(dev, "Select smem alloc_devs(0x%pK)\n", alloc_devs[0]);
> > +	} else {
> > +		alloc_devs[0] = &cam_dev->pdev->dev;
> > +		dev_dbg(dev, "Select default alloc_devs(0x%pK)\n",
> > +			alloc_devs[0]);
> > +	}
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Validate initialized num_planes & size[0] */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	/* Initialize buffer queue & locks */
> > +	INIT_LIST_HEAD(&node->pending_list);
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> 
> Aren't these initialized in mtk_cam_mem2mem2_v4l2_register?
> 

Yes, we will remove these initialization and only keep them in
mtk_cam_mem2mem2_v4l2_register function.

> > +
> > +	return 0;
> > +}
> > +
> > +static bool
> > +mtk_cam_all_nodes_streaming(struct mtk_cam_dev *cam_dev,
> > +			    struct mtk_cam_video_device *except)
> > +{
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +
> > +		if (node == except)
> > +			continue;
> > +		if (node->enabled && !vb2_start_streaming_called(&node->vbq))
> > +			return false;
> > +	}
> > +
> > +	return true;
> > +}
> > +
> > +static void mtk_cam_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +				       struct mtk_cam_video_device *node,
> > +				       enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	unsigned int i;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	for (i = 0; i < node->vbq.num_buffers; ++i)
> > +		if (node->vbq.bufs[i]->state == VB2_BUF_STATE_ACTIVE)
> > +			vb2_buffer_done(node->vbq.bufs[i], state);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d is not enable\n",
> > +			node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d %s failed\n",
> > +			node->id, __func__);
> > +		goto fail_return_bufs;
> > +	}
> > +
> > +	if (!mtk_cam_all_nodes_streaming(cam_dev, node))
> > +		return 0;
> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret < 0) {
> > +		dev_err(&cam_dev->pdev->dev, "Node:%d s_stream failed\n",
> > +			node->id);
> > +		goto fail_stop_pipeline;
> > +	}
> > +	return 0;
> > +
> > +fail_stop_pipeline:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_return_bufs:
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +
> > +	/* Was this the first node with streaming disabled? */
> > +	if (mtk_cam_all_nodes_streaming(cam_dev, node)) {
> > +		/* Yes, really stop streaming now */
> > +		if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +			dev_err(&cam_dev->pdev->dev,
> > +				"failed to stop streaming\n");
> > +	}
> > +	mtk_cam_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_try_fmt(struct file *file, void *fh,
> > +			   struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> > +
> > +	if (in_fmt->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc,
> > +					     in_fmt);
> > +	}
> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> > +	mtk_cam_dev_cal_mplane_pix_fmt(&cam_dev->pdev->dev,
> > +				       &in_fmt->fmt.pix_mp,
> > +				       node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_videoc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_fmt_set_img(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input)
> > +{
> > +	if (input->index > 0)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_CTRL:
> > +		return v4l2_ctrl_subscribe_event(fh, sub);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	if (node->id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = IMG_MAX_WIDTH;
> > +		sizes->stepwise.min_width = IMG_MIN_WIDTH;
> > +		sizes->stepwise.max_height = IMG_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = IMG_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	} else if (node->id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		sizes->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > +		sizes->stepwise.max_width = RRZ_MAX_WIDTH;
> > +		sizes->stepwise.min_width = RRZ_MIN_WIDTH;
> > +		sizes->stepwise.max_height = RRZ_MAX_HEIGHT;
> > +		sizes->stepwise.min_height = RRZ_MIN_HEIGHT;
> > +		sizes->stepwise.step_height = 1;
> > +		sizes->stepwise.step_width = 1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->index > 0 || f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	/* Each node is dedicated to only one meta format */
> > +	if (f->type != node->vbq.type)
> > +		return -EINVAL;
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +/* subdev internal operations */
> > +static const struct v4l2_subdev_internal_ops mtk_cam_subdev_internal_ops = {
> > +	.open = mtk_cam_subdev_open,
> > +	.close = mtk_cam_subdev_close,
> > +};
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_subdev_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream = mtk_cam_subdev_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> > +	.link_setup = mtk_cam_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> > +}
> 
> Move this function up with the other mtk_cam_vb2_* functions.
> 

Fix in next patch.

> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +/*
> > + * Config node's video properties
> > + * according to the device context requirement
> > + */
> > +static void mtk_cam_node_to_v4l2(struct mtk_cam_dev *cam_dev,
> > +				 unsigned int node,
> > +				 struct video_device *vdev,
> > +				 struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev_node_desc *node_desc =
> > +		&cam_dev->mem2mem2_nodes[node].desc;
> > +
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = V4L2_CAP_STREAMING | node_desc->cap;
> > +	f->type = node_desc->buf_type;
> > +	vdev->ioctl_ops = node_desc->ioctl_ops;
> > +
> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     node_desc,
> > +				     f);
> > +}
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = vb2_request_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	int ret;
> > +
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, media_dev);
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_register(struct device *dev,
> > +			  struct media_device *media_dev,
> > +			  struct v4l2_device *v4l2_dev,
> > +			  struct v4l2_ctrl_handler *ctrl_handler)
> 
> This can be a static function.
> 

After reveiw, we will remove this function and move the source code to
the mtk_cam_mem2mem2_v4l2_register. Below error handling will be
removed, either.

> > +{
> > +	int ret;
> > +
> > +	/* Set up v4l2 device */
> > +	v4l2_dev->ctrl_handler = ctrl_handler;
> > +	v4l2_dev->mdev = media_dev;
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", v4l2_dev);
> > +	ret = v4l2_device_register(dev, v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device (%d)\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_v4l2_dev:
> > +	media_device_unregister(media_dev);
> > +	media_device_cleanup(media_dev);
> 
> The calling function will do this cleanup when this function fails, so
> no need to do it here; just return ret.
> 

Ditto.


> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int num_nodes = cam_dev->dev_node_num;
> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_DEV_NODES + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		goto fail_media_dev;
> > +	}
> > +
> > +	ret = mtk_cam_v4l2_register(dev,
> > +				    &cam_dev->media_dev,
> > +				    &cam_dev->v4l2_dev,
> > +				    NULL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> > +	}
> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> 
> It doesn't look like this gets free'd in any of the failure cases below.
> 

Based on Tomasz comments, we could use devm_kcalloc() here and remove
any matching calls to kfree(), since that would be cleaned up when
removing the driver.
https://patchwork.kernel.org/patch/10905223/
Do I have misunderstanding?

> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_DEV_NODES].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +	cam_dev->subdev.internal_ops = &mtk_cam_subdev_internal_ops;
> > +
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < num_nodes; i++) {
> 
> Consider moving some of this loop to a new function. This would simplify
> the failure handling below by by removing the fail_vdev and
> fail_vdev_media_entity labels, whose cleanup could move into the helper
> function. It would also break up this large function a bit.
> 

Ok, thanks for your suggestion.
We will add new mtk_cam_video_register_device function to register video
device for each nodes and simplify the error handling.

	/* Create video nodes and links */
	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
		ret = mtk_cam_video_register_device(cam_dev, i);
		if (ret)
			goto fail_video_register;
	}

fail_video_register:
	i--;
	for (; i >= 0; i--) {
	video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
	media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
	mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
	}

> > +		struct mtk_cam_video_device *node = &cam_dev->mem2mem2_nodes[i];
> > +		struct video_device *vdev = &node->vdev;
> > +		struct vb2_queue *vbq = &node->vbq;
> > +		u32 output = !cam_dev->mem2mem2_nodes[i].desc.capture;
> > +		u32 link_flags = cam_dev->mem2mem2_nodes[i].desc.link_flags;
> > +
> > +		cam_dev->subdev_pads[i].flags = output ?
> > +			MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +		/* Initialize miscellaneous variables */
> > +		mutex_init(&node->lock);
> > +		spin_lock_init(&node->slock);
> > +		INIT_LIST_HEAD(&node->pending_list);
> > +
> > +		/* Initialize formats to default values */
> > +		mtk_cam_node_to_v4l2(cam_dev, i, vdev, &node->vdev_fmt);
> > +
> > +		/* Initialize media entities */
> > +		ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +		if (ret) {
> > +			dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +			goto fail_vdev_media_entity;
> > +		}
> > +		node->enabled = false;
> > +		node->id = i;
> > +		node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> > +		vdev->entity.ops = NULL;
> > +
> > +		/* Initialize vbq */
> > +		vbq->type = node->vdev_fmt.type;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +			vbq->io_modes = VB2_MMAP;
> > +		else
> > +			vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +		if (node->desc.smem_alloc)
> > +			vbq->bidirectional = 1;
> > +		if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +			vdev->entity.function =
> > +				MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +		vbq->ops = &mtk_cam_vb2_ops;
> > +		vbq->mem_ops = &vb2_dma_contig_memops;
> > +		vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +		vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +		vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +		/* Put the process hub sub device in the vb2 private data */
> > +		vbq->drv_priv = cam_dev;
> > +		vbq->lock = &node->lock;
> > +		vbq->supports_requests = true;
> > +
> > +		ret = vb2_queue_init(vbq);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Initialize vdev */
> > +		snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +			 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +		vdev->release = video_device_release_empty;
> > +		vdev->fops = &mtk_cam_v4l2_fops;
> > +		vdev->lock = &node->lock;
> > +		vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +		vdev->queue = &node->vbq;
> > +		vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +		/* Enable private control for image video devices */
> > +		if (node->desc.image) {
> > +			mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +			vdev->ctrl_handler = &node->ctrl_handler;
> > +		}
> > +		video_set_drvdata(vdev, cam_dev);
> > +		dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +		if (ret) {
> > +			dev_err(dev, "failed to register vde:%d\n", ret);
> > +			goto fail_vdev;
> > +		}
> > +
> > +		/* Create link between video node and the subdev pad */
> > +		if (output) {
> > +			ret = media_create_pad_link(&vdev->entity, 0,
> > +						    &cam_dev->subdev.entity,
> > +						    i, link_flags);
> > +		} else {
> > +			ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +						    i, &vdev->entity, 0,
> > +						    link_flags);
> > +		}
> > +		if (ret)
> > +			goto fail_link;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +	for (; i >= 0; i--) {
> > +fail_link:
> > +		video_unregister_device(&cam_dev->mem2mem2_nodes[i].vdev);
> > +fail_vdev:
> > +		media_entity_cleanup(&cam_dev->mem2mem2_nodes[i].vdev.entity);
> > +fail_vdev_media_entity:
> > +		mutex_destroy(&cam_dev->mem2mem2_nodes[i].lock);
> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +fail_media_dev:
> >
> media_device_unregister and media_device_cleanup are called when
> mtk_cam_media_register fails, so only need to call these under the
> fail_v4l2_dev label.
> 

Will correct in next patch.

> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> > +
> > +	for (i = 0; i < cam_dev->dev_node_num; i++) {
> > +		dev = &cam_dev->mem2mem2_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_SENINF_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_P1_HUB_PAD_SINK,
> > +				    0);
> > +	if (ret)
> 
> should this function return an error here?
> 

Ok, we will print error log and just return here.

> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> 
> Should anything be done to cam_dev-seninf here, such as setting it to
> NULL?
> 

Ok, we will configure cam_dev-seninf to NULL here in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_dev_fwnode_parse(struct device *dev,
> > +				    struct v4l2_fwnode_endpoint *vep,
> > +				    struct v4l2_async_subdev *asd)
> > +{
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(
> > +		&cam_dev->pdev->dev, &cam_dev->notifier,
> > +		sizeof(struct v4l2_async_subdev),
> > +		mtk_cam_dev_fwnode_parse);
> 
> As far as I can tell, the fourth argument is optional, so I think you
> can just set NULL and remove mtk_cam_dev_fwnode_parse.
> 

Good point. Thanks for your suggestion.
Will fix in next patch.

> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > new file mode 100644
> > index 000000000000..73b36916da08
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > @@ -0,0 +1,43 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Frederic Chen <frederic.chen@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <media/v4l2-device.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +int mtk_cam_videoc_querycap(struct file *file, void *fh,
> > +			    struct v4l2_capability *cap);
> > +int mtk_cam_enum_framesizes(struct file *filp, void *priv,
> > +			    struct v4l2_frmsizeenum *sizes);
> > +int mtk_cam_videoc_enum_fmt(struct file *file, void *fh,
> > +			    struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
> > +int mtk_cam_videoc_try_fmt(struct file *file,
> > +			   void *fh, struct v4l2_format *in_fmt);
> > +int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +			      struct v4l2_input *input);
> > +int mtk_cam_vidioc_g_input(struct file *file, void *fh, unsigned int *input);
> > +int mtk_cam_vidioc_s_input(struct file *file, void *fh, unsigned int input);
> > +int mtk_cam_meta_enum_format(struct file *file, void *fh,
> > +			     struct v4l2_fmtdesc *f);
> > +int mtk_cam_videoc_g_meta_fmt(struct file *file, void *fh,
> > +			      struct v4l2_format *f);
> > +int mtk_cam_vidioc_subscribe_event(struct v4l2_fh *fh,
> > +				   const struct v4l2_event_subscription *sub);
> > +
> > +#endif /* __MTK_CAM_DEV_V4L2_H__ */
> > -- 
> > 2.18.0
> > 

Best regards,


Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, V3 0/9] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-06-11  3:53   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 is Kconfig configuration for ISP P1 driver.
* Patch 4 extends the original V4L2 image & meta formats.
* Patch 5 add Mediatek specific v4l2 control ID for ISP P1 driver.
* Patch 6 provides V4L2 utility functions & default video devices
  configuration & initialization for ISP P1 driver.
* Patch 7 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
* Patch 8 adds communication with the co-processor on the SoC through
  the SCP driver[2].
* Patch 9 provides ISP P1 shard memory management between ISP P1 &
  co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch[1][4][5]:


/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.43

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.43

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
    pad0: Sink
        <- "MK-ISP-P1-V4L2 meta input":0 []
    pad1: Source
        -> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
    pad2: Source
        -> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
    pad3: Source
        -> "MTK-ISP-P1-V4L2 partial meta 0":0 []
    pad4: Source
        -> "MTK-ISP-P1-V4L2 partial meta 1":0 []
    pad5: Source
        -> "MTK-ISP-P1-V4L2 partial meta 2":0 []
    pad6: Source
        -> "MTK-ISP-P1-V4L2 partial meta 3":0 []
    pad7: Source
    pad8: Source
    pad9: Source
    pad10: Source
    pad11: Sink
        <- "1a040000.seninf.mipi-csi":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video12
    pad0: Sink
        -> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video13
    pad0: Source
        <- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
            device node name /dev/video14
    pad0: Source
        <- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
    pad0: Source
        <- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
    pad0: Source
        <- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
    pad0: Source
        <- "MTK-ISP-P1-V4L2":5 []

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
    pad0: Source
        <- "MTK-ISP-P1-V4L2":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
    pad0: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov5695 2-0036":0 []
    pad1: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov2685 4-003c":0 []
    pad2: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad3: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad4: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "MTK-ISP-P1-V4L2":11 []
    pad5: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad6: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad7: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad8: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad9: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad10: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad11: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
    pad0: Source
        [fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
    pad0: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10972143/

[3]. [RFC,V1,6/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.kernel.org/patch/10905223/

[4]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[5]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
This testing depends on the above sensors[4][5] patches.

Note 2.
Before passing streaming on testing, need to enable media links between
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i -d /dev/media1
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media1:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43

Required ioctls:
    test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
    test second /dev/media1 open: OK
    test MEDIA_IOC_DEVICE_INFO: OK
    test for unlimited opens: OK

Media Controller ioctls:
    test MEDIA_IOC_G_TOPOLOGY: OK
    Entities: 11 Interfaces: 11 Pads: 33 Links: 21
    test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
    test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video12:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x8c200000
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x0c200000
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000010
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000e (14)
    Name             : MTK-ISP-P1-V4L2 meta input
    Function         : V4L2 I/O
    Pad 0x0100000f   : 0: Sink
      Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video12 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video12: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video13:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000016
    Type             : V4L Video
Entity Info:
    ID               : 0x00000014 (20)
    Name             : MTK-ISP-P1-V4L2 main stream
    Function         : V4L2 I/O
    Pad 0x01000015   : 0: Source
      Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video13 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video13: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video14:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300001c
    Type             : V4L Video
Entity Info:
    ID               : 0x0000001a (26)
    Name             : MTK-ISP-P1-V4L2 packed out
    Function         : V4L2 I/O
    Pad 0x0100001b   : 0: Source
      Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video14 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video14: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000022
    Type             : V4L Video
Entity Info:
    ID               : 0x00000020 (32)
    Name             : MTK-ISP-P1-V4L2 partial meta 0
    Function         : V4L2 I/O
    Pad 0x01000021   : 0: Source
      Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video15 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000028
    Type             : V4L Video
Entity Info:
    ID               : 0x00000026 (38)
    Name             : MTK-ISP-P1-V4L2 partial meta 1
    Function         : V4L2 I/O
    Pad 0x01000027   : 0: Source
      Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video16 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300002e
    Type             : V4L Video
Entity Info:
    ID               : 0x0000002c (44)
    Name             : MTK-ISP-P1-V4L2 partial meta 2
    Function         : V4L2 I/O
    Pad 0x0100002d   : 0: Source
      Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video17 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000034
    Type             : V4L Video
Entity Info:
    ID               : 0x00000032 (50)
    Name             : MTK-ISP-P1-V4L2 partial meta 3
    Function         : V4L2 I/O
    Pad 0x01000033   : 0: Source
      Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video18 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300004f
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000001 (1)
    Name             : MTK-ISP-P1-V4L2
    Function         : Video Statistics
    Pad 0x01000002   : 0: Sink
      Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
    Pad 0x01000003   : 1: Source
      Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
    Pad 0x01000004   : 2: Source
      Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
    Pad 0x01000005   : 3: Source
      Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
    Pad 0x01000006   : 4: Source
      Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
    Pad 0x01000007   : 5: Source
      Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data
    Pad 0x01000008   : 6: Source
      Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data
    Pad 0x01000009   : 7: Source
    Pad 0x0100000a   : 8: Source
    Pad 0x0100000b   : 9: Source
    Pad 0x0100000c   : 10: Source
    Pad 0x0100000d   : 11: Sink
      Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev5 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000051
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000038 (56)
    Name             : 1a040000.seninf.mipi-csi
    Function         : Video Interface Bridge
    Pad 0x01000039   : 0: Sink
      Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
    Pad 0x0100003a   : 1: Sink
      Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
    Pad 0x0100003b   : 2: Sink
    Pad 0x0100003c   : 3: Sink
    Pad 0x0100003d   : 4: Source
      Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
    Pad 0x0100003e   : 5: Source
    Pad 0x0100003f   : 6: Source
    Pad 0x01000040   : 7: Source
    Pad 0x01000041   : 8: Source
    Pad 0x01000042   : 9: Source
    Pad 0x01000043   : 10: Source
    Pad 0x01000044   : 11: Source

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev6 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 2 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000053
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000045 (69)
    Name             : ov5695 2-0036
    Function         : Camera Sensor
    Pad 0x01000046   : 0: Source
      Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev7 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 11 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev8:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000055
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000049 (73)
    Name             : ov2685 4-003c
    Function         : Camera Sensor
    Pad 0x0100004a   : 0: Source
      Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev8 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 10 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media1: 668, Succeeded: 668, Failed: 0, Warnings: 0

--------------------------------------------------------------------------------

Jungo Lin (9):
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   57 +
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst  |   49 +
 Documentation/media/uapi/v4l/pixfmt-mtba.rst  |   62 +
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst  |   58 +
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst  |   70 +
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst  |   75 +
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst  |   87 +
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst  |  107 ++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst  |  107 ++
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   24 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 drivers/media/platform/mtk-isp/Makefile       |    3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    9 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  138 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   38 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  371 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 ++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         |  304 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   18 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  251 +++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   13 +
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   17 +
 29 files changed, 5153 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,V3 0/9] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-06-11  3:53   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 is Kconfig configuration for ISP P1 driver.
* Patch 4 extends the original V4L2 image & meta formats.
* Patch 5 add Mediatek specific v4l2 control ID for ISP P1 driver.
* Patch 6 provides V4L2 utility functions & default video devices
  configuration & initialization for ISP P1 driver.
* Patch 7 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
* Patch 8 adds communication with the co-processor on the SoC through
  the SCP driver[2].
* Patch 9 provides ISP P1 shard memory management between ISP P1 &
  co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch[1][4][5]:


/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.43

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.43

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
    pad0: Sink
        <- "MK-ISP-P1-V4L2 meta input":0 []
    pad1: Source
        -> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
    pad2: Source
        -> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
    pad3: Source
        -> "MTK-ISP-P1-V4L2 partial meta 0":0 []
    pad4: Source
        -> "MTK-ISP-P1-V4L2 partial meta 1":0 []
    pad5: Source
        -> "MTK-ISP-P1-V4L2 partial meta 2":0 []
    pad6: Source
        -> "MTK-ISP-P1-V4L2 partial meta 3":0 []
    pad7: Source
    pad8: Source
    pad9: Source
    pad10: Source
    pad11: Sink
        <- "1a040000.seninf.mipi-csi":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video12
    pad0: Sink
        -> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video13
    pad0: Source
        <- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
            device node name /dev/video14
    pad0: Source
        <- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
    pad0: Source
        <- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
    pad0: Source
        <- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
    pad0: Source
        <- "MTK-ISP-P1-V4L2":5 []

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
    pad0: Source
        <- "MTK-ISP-P1-V4L2":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
    pad0: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov5695 2-0036":0 []
    pad1: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov2685 4-003c":0 []
    pad2: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad3: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad4: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "MTK-ISP-P1-V4L2":11 []
    pad5: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad6: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad7: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad8: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad9: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad10: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad11: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
    pad0: Source
        [fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
    pad0: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10972143/

[3]. [RFC,V1,6/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.kernel.org/patch/10905223/

[4]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[5]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
This testing depends on the above sensors[4][5] patches.

Note 2.
Before passing streaming on testing, need to enable media links between
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i -d /dev/media1
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media1:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43

Required ioctls:
    test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
    test second /dev/media1 open: OK
    test MEDIA_IOC_DEVICE_INFO: OK
    test for unlimited opens: OK

Media Controller ioctls:
    test MEDIA_IOC_G_TOPOLOGY: OK
    Entities: 11 Interfaces: 11 Pads: 33 Links: 21
    test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
    test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video12:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x8c200000
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x0c200000
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000010
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000e (14)
    Name             : MTK-ISP-P1-V4L2 meta input
    Function         : V4L2 I/O
    Pad 0x0100000f   : 0: Sink
      Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video12 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video12: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video13:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000016
    Type             : V4L Video
Entity Info:
    ID               : 0x00000014 (20)
    Name             : MTK-ISP-P1-V4L2 main stream
    Function         : V4L2 I/O
    Pad 0x01000015   : 0: Source
      Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video13 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video13: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video14:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300001c
    Type             : V4L Video
Entity Info:
    ID               : 0x0000001a (26)
    Name             : MTK-ISP-P1-V4L2 packed out
    Function         : V4L2 I/O
    Pad 0x0100001b   : 0: Source
      Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video14 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video14: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000022
    Type             : V4L Video
Entity Info:
    ID               : 0x00000020 (32)
    Name             : MTK-ISP-P1-V4L2 partial meta 0
    Function         : V4L2 I/O
    Pad 0x01000021   : 0: Source
      Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video15 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000028
    Type             : V4L Video
Entity Info:
    ID               : 0x00000026 (38)
    Name             : MTK-ISP-P1-V4L2 partial meta 1
    Function         : V4L2 I/O
    Pad 0x01000027   : 0: Source
      Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video16 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300002e
    Type             : V4L Video
Entity Info:
    ID               : 0x0000002c (44)
    Name             : MTK-ISP-P1-V4L2 partial meta 2
    Function         : V4L2 I/O
    Pad 0x0100002d   : 0: Source
      Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video17 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000034
    Type             : V4L Video
Entity Info:
    ID               : 0x00000032 (50)
    Name             : MTK-ISP-P1-V4L2 partial meta 3
    Function         : V4L2 I/O
    Pad 0x01000033   : 0: Source
      Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video18 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300004f
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000001 (1)
    Name             : MTK-ISP-P1-V4L2
    Function         : Video Statistics
    Pad 0x01000002   : 0: Sink
      Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
    Pad 0x01000003   : 1: Source
      Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
    Pad 0x01000004   : 2: Source
      Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
    Pad 0x01000005   : 3: Source
      Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
    Pad 0x01000006   : 4: Source
      Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
    Pad 0x01000007   : 5: Source
      Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data
    Pad 0x01000008   : 6: Source
      Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data
    Pad 0x01000009   : 7: Source
    Pad 0x0100000a   : 8: Source
    Pad 0x0100000b   : 9: Source
    Pad 0x0100000c   : 10: Source
    Pad 0x0100000d   : 11: Sink
      Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev5 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000051
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000038 (56)
    Name             : 1a040000.seninf.mipi-csi
    Function         : Video Interface Bridge
    Pad 0x01000039   : 0: Sink
      Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
    Pad 0x0100003a   : 1: Sink
      Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
    Pad 0x0100003b   : 2: Sink
    Pad 0x0100003c   : 3: Sink
    Pad 0x0100003d   : 4: Source
      Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
    Pad 0x0100003e   : 5: Source
    Pad 0x0100003f   : 6: Source
    Pad 0x01000040   : 7: Source
    Pad 0x01000041   : 8: Source
    Pad 0x01000042   : 9: Source
    Pad 0x01000043   : 10: Source
    Pad 0x01000044   : 11: Source

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev6 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 2 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000053
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000045 (69)
    Name             : ov5695 2-0036
    Function         : Camera Sensor
    Pad 0x01000046   : 0: Source
      Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev7 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 11 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev8:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000055
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000049 (73)
    Name             : ov2685 4-003c
    Function         : Camera Sensor
    Pad 0x0100004a   : 0: Source
      Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev8 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 10 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media1: 668, Succeeded: 668, Failed: 0, Warnings: 0

--------------------------------------------------------------------------------

Jungo Lin (9):
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   57 +
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst  |   49 +
 Documentation/media/uapi/v4l/pixfmt-mtba.rst  |   62 +
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst  |   58 +
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst  |   70 +
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst  |   75 +
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst  |   87 +
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst  |  107 ++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst  |  107 ++
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   24 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 drivers/media/platform/mtk-isp/Makefile       |    3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    9 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  138 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   38 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  371 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 ++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         |  304 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   18 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  251 +++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   13 +
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   17 +
 29 files changed, 5153 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0


^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, V3 0/9] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-06-11  3:53   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which
will be used in camera features of CrOS. It's the first time Mediatek
develops ISP kernel drivers based on V4L2 and media controller
framework. I posted the main part of the ISP Pass 1 driver as RFC to
discuss first and would like some review comments on the overall
architecture of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 is Kconfig configuration for ISP P1 driver.
* Patch 4 extends the original V4L2 image & meta formats.
* Patch 5 add Mediatek specific v4l2 control ID for ISP P1 driver.
* Patch 6 provides V4L2 utility functions & default video devices
  configuration & initialization for ISP P1 driver.
* Patch 7 is the heart of ISP P1 driver. It handles the ISP
  HW configuration, provides interrupt handling and initializes
  the V4L2 device nodes and other functions.
* Patch 8 adds communication with the co-processor on the SoC through
  the SCP driver[2].
* Patch 9 provides ISP P1 shard memory management between ISP P1 &
  co-processor. It is controlled by child device of ISP P1 driver.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are
implemented in below patch[1][4][5]:


/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.43

Media device information
------------------------
driver          mtk-cam
model           MTK-ISP-P1-V4L2
serial
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.43

Device topology
- entity 1: MTK-ISP-P1-V4L2 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
    pad0: Sink
        <- "MK-ISP-P1-V4L2 meta input":0 []
    pad1: Source
        -> "MTK-ISP-P1-V4L2 main stream":0 [ENABLED]
    pad2: Source
        -> "MTK-ISP-P1-V4L2 packed out":0 [ENABLED]
    pad3: Source
        -> "MTK-ISP-P1-V4L2 partial meta 0":0 []
    pad4: Source
        -> "MTK-ISP-P1-V4L2 partial meta 1":0 []
    pad5: Source
        -> "MTK-ISP-P1-V4L2 partial meta 2":0 []
    pad6: Source
        -> "MTK-ISP-P1-V4L2 partial meta 3":0 []
    pad7: Source
    pad8: Source
    pad9: Source
    pad10: Source
    pad11: Sink
        <- "1a040000.seninf.mipi-csi":4 []

- entity 14: MTK-ISP-P1-V4L2 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video12
    pad0: Sink
        -> "MTK-ISP-P1-V4L2":0 []

- entity 20: MTK-ISP-P1-V4L2 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video13
    pad0: Source
        <- "MTK-ISP-P1-V4L2":1 [ENABLED]

- entity 26: MTK-ISP-P1-V4L2 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
            device node name /dev/video14
    pad0: Source
        <- "MTK-ISP-P1-V4L2":2 [ENABLED]

- entity 32: MTK-ISP-P1-V4L2 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video15
    pad0: Source
        <- "MTK-ISP-P1-V4L2":3 []

- entity 38: MTK-ISP-P1-V4L2 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video16
    pad0: Source
        <- "MTK-ISP-P1-V4L2":4 []

- entity 44: MTK-ISP-P1-V4L2 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video17
    pad0: Source
        <- "MTK-ISP-P1-V4L2":5 []

- entity 50: MTK-ISP-P1-V4L2 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
    pad0: Source
        <- "MTK-ISP-P1-V4L2":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
    pad0: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov5695 2-0036":0 []
    pad1: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        <- "ov2685 4-003c":0 []
    pad2: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad3: Sink
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad4: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "MTK-ISP-P1-V4L2":11 []
    pad5: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad6: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad7: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad8: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad9: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad10: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
    pad11: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
    pad0: Source
        [fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
    pad0: Source
        [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
        -> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/

[2]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/10972143/

[3]. [RFC,V1,6/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.kernel.org/patch/10905223/

[4]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[5]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/utils/v4l2-compliance?id=2aff3ef768c42cfdbb31d143ee2286a6b46e9db0

Note 1.
This testing depends on the above sensors[4][5] patches.

Note 2.
Before passing streaming on testing, need to enable media links between
entity 1: MTK-ISP-P1-V4L2, entity 20: MTK-ISP-P1-V4L2 main stream
and entity 26: MTK-ISP-P1-V4L2 packed out.

/usr/bin/media-ctl -i -d /dev/media1
Enter a link to modify or enter to stop
1:1->20:0[1]
1:1->20:0[1]
Enter a link to modify or enter to stop
1:2->26:0[1]
1:2->26:0[1]
Enter a link to modify or enter to stop

v4l2-compliance test output:
/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam device /dev/media1:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43

Required ioctls:
    test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
    test second /dev/media1 open: OK
    test MEDIA_IOC_DEVICE_INFO: OK
    test for unlimited opens: OK

Media Controller ioctls:
    test MEDIA_IOC_G_TOPOLOGY: OK
    Entities: 11 Interfaces: 11 Pads: 33 Links: 21
    test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
    test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video12:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x8c200000
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x0c200000
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000010
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000e (14)
    Name             : MTK-ISP-P1-V4L2 meta input
    Function         : V4L2 I/O
    Pad 0x0100000f   : 0: Sink
      Link 0x02000012: to remote pad 0x1000002 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video12 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video12: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video13:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000016
    Type             : V4L Video
Entity Info:
    ID               : 0x00000014 (20)
    Name             : MTK-ISP-P1-V4L2 main stream
    Function         : V4L2 I/O
    Pad 0x01000015   : 0: Source
      Link 0x02000018: from remote pad 0x1000003 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video13 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video13: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video14:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300001c
    Type             : V4L Video
Entity Info:
    ID               : 0x0000001a (26)
    Name             : MTK-ISP-P1-V4L2 packed out
    Function         : V4L2 I/O
    Pad 0x0100001b   : 0: Source
      Link 0x0200001e: from remote pad 0x1000004 of entity 'MTK-ISP-P1-V4L2': Data, Enabled

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video14 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 3

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video14: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video15:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000022
    Type             : V4L Video
Entity Info:
    ID               : 0x00000020 (32)
    Name             : MTK-ISP-P1-V4L2 partial meta 0
    Function         : V4L2 I/O
    Pad 0x01000021   : 0: Source
      Link 0x02000024: from remote pad 0x1000005 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video15 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video15: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video16:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000028
    Type             : V4L Video
Entity Info:
    ID               : 0x00000026 (38)
    Name             : MTK-ISP-P1-V4L2 partial meta 1
    Function         : V4L2 I/O
    Pad 0x01000027   : 0: Source
      Link 0x0200002a: from remote pad 0x1000006 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video16 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video16: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video17:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300002e
    Type             : V4L Video
Entity Info:
    ID               : 0x0000002c (44)
    Name             : MTK-ISP-P1-V4L2 partial meta 2
    Function         : V4L2 I/O
    Pad 0x0100002d   : 0: Source
      Link 0x02000030: from remote pad 0x1000007 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video17 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video17: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for MTK-ISP-P1-V4L2 device /dev/video18:

Driver Info:
    Driver name      : MTK-ISP-P1-V4L2
    Card type        : MTK-ISP-P1-V4L2
    Bus info         : platform:1a000000.camisp
    Driver version   : 4.19.43
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000034
    Type             : V4L Video
Entity Info:
    ID               : 0x00000032 (50)
    Name             : MTK-ISP-P1-V4L2 partial meta 3
    Function         : V4L2 I/O
    Pad 0x01000033   : 0: Source
      Link 0x02000036: from remote pad 0x1000008 of entity 'MTK-ISP-P1-V4L2': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
    test second /dev/video18 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK

Total for MTK-ISP-P1-V4L2 device /dev/video18: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev5:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x0300004f
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000001 (1)
    Name             : MTK-ISP-P1-V4L2
    Function         : Video Statistics
    Pad 0x01000002   : 0: Sink
      Link 0x02000012: from remote pad 0x100000f of entity 'MTK-ISP-P1-V4L2 meta input': Data
    Pad 0x01000003   : 1: Source
      Link 0x02000018: to remote pad 0x1000015 of entity 'MTK-ISP-P1-V4L2 main stream': Data, Enabled
    Pad 0x01000004   : 2: Source
      Link 0x0200001e: to remote pad 0x100001b of entity 'MTK-ISP-P1-V4L2 packed out': Data, Enabled
    Pad 0x01000005   : 3: Source
      Link 0x02000024: to remote pad 0x1000021 of entity 'MTK-ISP-P1-V4L2 partial meta 0': Data
    Pad 0x01000006   : 4: Source
      Link 0x0200002a: to remote pad 0x1000027 of entity 'MTK-ISP-P1-V4L2 partial meta 1': Data
    Pad 0x01000007   : 5: Source
      Link 0x02000030: to remote pad 0x100002d of entity 'MTK-ISP-P1-V4L2 partial meta 2': Data
    Pad 0x01000008   : 6: Source
      Link 0x02000036: to remote pad 0x1000033 of entity 'MTK-ISP-P1-V4L2 partial meta 3': Data
    Pad 0x01000009   : 7: Source
    Pad 0x0100000a   : 8: Source
    Pad 0x0100000b   : 9: Source
    Pad 0x0100000c   : 10: Source
    Pad 0x0100000d   : 11: Sink
      Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev5 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
    test VIDIOC_QUERYCTRL: OK (Not Supported)
    test VIDIOC_G/S_CTRL: OK (Not Supported)
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 0 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev6:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000051
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000038 (56)
    Name             : 1a040000.seninf.mipi-csi
    Function         : Video Interface Bridge
    Pad 0x01000039   : 0: Sink
      Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
    Pad 0x0100003a   : 1: Sink
      Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
    Pad 0x0100003b   : 2: Sink
    Pad 0x0100003c   : 3: Sink
    Pad 0x0100003d   : 4: Source
      Link 0x0200004d: to remote pad 0x100000d of entity 'MTK-ISP-P1-V4L2': Data
    Pad 0x0100003e   : 5: Source
    Pad 0x0100003f   : 6: Source
    Pad 0x01000040   : 7: Source
    Pad 0x01000041   : 8: Source
    Pad 0x01000042   : 9: Source
    Pad 0x01000043   : 10: Source
    Pad 0x01000044   : 11: Source

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev6 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 2 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev6: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev7:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000053
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000045 (69)
    Name             : ov5695 2-0036
    Function         : Camera Sensor
    Pad 0x01000046   : 0: Source
      Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev7 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 11 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam device /dev/v4l-subdev8:

Media Driver Info:
    Driver name      : mtk-cam
    Model            : MTK-ISP-P1-V4L2
    Serial           : 
    Bus info         : platform:1a000000.camisp
    Media version    : 4.19.43
    Hardware revision: 0x00000000 (0)
    Driver version   : 4.19.43
Interface Info:
    ID               : 0x03000055
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000049 (73)
    Name             : ov2685 4-003c
    Function         : Camera Sensor
    Pad 0x0100004a   : 0: Source
      Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
    test second /dev/v4l-subdev8 open: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
    test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Try VIDIOC_SUBDEV_G/S_FMT: OK
    test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
    test Active VIDIOC_SUBDEV_G/S_FMT: OK
    test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
    test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 10 Private Controls: 0

Format ioctls:
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
    test VIDIOC_G/S_PARM: OK (Not Supported)
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK (Not Supported)
    test VIDIOC_TRY_FMT: OK (Not Supported)
    test VIDIOC_S_FMT: OK (Not Supported)
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls:
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
    test VIDIOC_EXPBUF: OK (Not Supported)
    test Requests: OK (Not Supported)

Total for mtk-cam device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam device /dev/media1: 668, Succeeded: 668, Failed: 0, Warnings: 0

--------------------------------------------------------------------------------

Jungo Lin (9):
  dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 control
  media: platform: Add Mediatek ISP P1 V4L2 functions
  media: platform: Add Mediatek ISP P1 device driver
  media: platform: Add Mediatek ISP P1 SCP communication
  media: platform: Add Mediatek ISP P1 shared memory device

 .../bindings/media/mediatek,camisp.txt        |   57 +
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst  |   49 +
 Documentation/media/uapi/v4l/pixfmt-mtba.rst  |   62 +
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst  |   58 +
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst  |   70 +
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst  |   75 +
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst  |   87 +
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst  |  107 ++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst  |  107 ++
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   24 +
 drivers/media/platform/Kconfig                |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 drivers/media/platform/mtk-isp/Makefile       |    3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    9 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         |  138 ++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |   38 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c |  371 ++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h |  207 ++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         |  304 +++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |   18 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  251 +++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   13 +
 include/uapi/linux/v4l2-controls.h            |    4 +
 include/uapi/linux/videodev2.h                |   17 +
 29 files changed, 5153 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,v3 1/9] dt-bindings: mt8183: Added camera ISP Pass 1
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 57 +++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..50a8b4d9ac8e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,57 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block registers and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camsys base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Single camera ISP P1 hardware module A
+  "cam_b": Single camera ISP P1 hardware module B
+- interrupts: Interrupt number to the CPU.
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
\ No newline at end of file
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 1/9] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 57 +++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..50a8b4d9ac8e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,57 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block registers and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camsys base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Single camera ISP P1 hardware module A
+  "cam_b": Single camera ISP P1 hardware module B
+- interrupts: Interrupt number to the CPU.
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
\ No newline at end of file
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 1/9] dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit in
Mediatek's camera ISP system. The Pass 1 unit grabs the sensor data
out from the sensor interface, applies ISP image effects from tuning
data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 57 +++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..50a8b4d9ac8e
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,57 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block registers and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camsys base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Single camera ISP P1 hardware module A
+  "cam_b": Single camera ISP P1 hardware module B
+- interrupts: Interrupt number to the CPU.
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
\ No newline at end of file
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 2/9] dts: arm64: mt8183: Add ISP Pass 1 nodes
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..8a725357c594 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -369,5 +369,29 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
+
 	};
 };
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 2/9] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..8a725357c594 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -369,5 +369,29 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
+
 	};
 };
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 2/9] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 75c4881bbe5e..8a725357c594 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -369,5 +369,29 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+			      <0 0x1a003000 0 0x1000>,
+			      <0 0x1a004000 0 0x2000>,
+			      <0 0x1a006000 0 0x2000>;
+			reg-names = "cam_sys",
+				    "cam_uni",
+				    "cam_a",
+				    "cam_b";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				     <GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>;
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+				 <&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+				      "camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			mediatek,scp = <&scp>;
+		};
+
 	};
 };
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 3/9] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 17 +++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 011c1c2fcf19..8e2b65d757e5 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..983b79c261fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	bool "Mediatek Pass 1 image processing function"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
+	select V4L2_FWNODE
+	select VIDEOBUF2_DMA_CONTIG
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
+
+
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 3/9] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 17 +++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 011c1c2fcf19..8e2b65d757e5 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..983b79c261fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	bool "Mediatek Pass 1 image processing function"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
+	select V4L2_FWNODE
+	select VIDEOBUF2_DMA_CONTIG
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
+
+
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 3/9] media: platform: Add Mediatek ISP Pass 1 driver Kconfig
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds Kconfig for Pass 1 (P1) unit driver of Mediatek's
camera ISP system. ISP P1 unit is embedded in Mediatek SoCs. It
provides RAW processing which includes optical black correction,
defect pixel correction, W/IR imbalance correction and lens
shading correction.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/Kconfig         |  2 ++
 drivers/media/platform/mtk-isp/Kconfig | 17 +++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 011c1c2fcf19..8e2b65d757e5 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -32,6 +32,8 @@ source "drivers/media/platform/davinci/Kconfig"
 
 source "drivers/media/platform/omap/Kconfig"
 
+source "drivers/media/platform/mtk-isp/Kconfig"
+
 config VIDEO_ASPEED
 	tristate "Aspeed AST2400 and AST2500 Video Engine driver"
 	depends on VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..983b79c261fa
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	bool "Mediatek Pass 1 image processing function"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
+	select V4L2_FWNODE
+	select VIDEOBUF2_DMA_CONTIG
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
+
+
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v3 4/9] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst |  49 +++++++++
 Documentation/media/uapi/v4l/pixfmt-mtba.rst |  62 +++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst |  58 ++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst |  70 ++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst |  75 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst |  87 +++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst | 107 +++++++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst | 107 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c         |  13 +++
 include/uapi/linux/videodev2.h               |  17 +++
 10 files changed, 645 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtb8.rst b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
new file mode 100644
index 000000000000..2337ccd66277
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
@@ -0,0 +1,49 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B8 ('MTB8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other in memory, with a depth of 8 bits per pixel.
+Each sample is stored in a byte.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtba.rst b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
new file mode 100644
index 000000000000..ade51d5472b0
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
@@ -0,0 +1,62 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B10 ('MTBA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbc.rst b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
new file mode 100644
index 000000000000..b122600fddb5
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
@@ -0,0 +1,58 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B12 ('MTBC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbe.rst b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
new file mode 100644
index 000000000000..4b9bc9a62504
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
@@ -0,0 +1,70 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B14 ('MTBE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtf8.rst b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
new file mode 100644
index 000000000000..51c9ddc4e20d
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
@@ -0,0 +1,75 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F8 ('MTF8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 8 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfa.rst b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
new file mode 100644
index 000000000000..68421c44f5e7
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
@@ -0,0 +1,87 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F10 ('MTFA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 10 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfc.rst b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
new file mode 100644
index 000000000000..a3535f5435fa
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F12 ('MTFC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 12 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfe.rst b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
new file mode 100644
index 000000000000..324a258e897f
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F14 ('MTFE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 14 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index ac87c3e37280..2f536fedd9c4 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1297,6 +1297,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_B8:	descr = "8-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F8:	descr = "8-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B10:	descr = "10-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F10:	descr = "10-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B12:	descr = "12-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F12:	descr = "12-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B14:	descr = "14-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F14:	descr = "14-bit Packed Full-G Bayer format"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1312,6 +1320,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local contrast enhanced statistics"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local motion vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP tuning metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1050a75fb7ef..ef51911fcfe4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,16 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10	v4l2_fourcc('M', 'T', 'B', 'A') /* Packed bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12	v4l2_fourcc('M', 'T', 'B', 'C') /* Packed bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14	v4l2_fourcc('M', 'T', 'B', 'E') /* Packed bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10	v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12	v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14	v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -750,6 +760,13 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
 
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS	v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A		v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF		v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
 
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 4/9] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst |  49 +++++++++
 Documentation/media/uapi/v4l/pixfmt-mtba.rst |  62 +++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst |  58 ++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst |  70 ++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst |  75 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst |  87 +++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst | 107 +++++++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst | 107 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c         |  13 +++
 include/uapi/linux/videodev2.h               |  17 +++
 10 files changed, 645 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtb8.rst b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
new file mode 100644
index 000000000000..2337ccd66277
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
@@ -0,0 +1,49 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B8 ('MTB8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other in memory, with a depth of 8 bits per pixel.
+Each sample is stored in a byte.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtba.rst b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
new file mode 100644
index 000000000000..ade51d5472b0
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
@@ -0,0 +1,62 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B10 ('MTBA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbc.rst b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
new file mode 100644
index 000000000000..b122600fddb5
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
@@ -0,0 +1,58 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B12 ('MTBC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbe.rst b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
new file mode 100644
index 000000000000..4b9bc9a62504
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
@@ -0,0 +1,70 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B14 ('MTBE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtf8.rst b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
new file mode 100644
index 000000000000..51c9ddc4e20d
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
@@ -0,0 +1,75 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F8 ('MTF8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 8 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfa.rst b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
new file mode 100644
index 000000000000..68421c44f5e7
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
@@ -0,0 +1,87 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F10 ('MTFA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 10 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfc.rst b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
new file mode 100644
index 000000000000..a3535f5435fa
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F12 ('MTFC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 12 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfe.rst b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
new file mode 100644
index 000000000000..324a258e897f
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F14 ('MTFE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 14 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index ac87c3e37280..2f536fedd9c4 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1297,6 +1297,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_B8:	descr = "8-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F8:	descr = "8-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B10:	descr = "10-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F10:	descr = "10-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B12:	descr = "12-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F12:	descr = "12-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B14:	descr = "14-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F14:	descr = "14-bit Packed Full-G Bayer format"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1312,6 +1320,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local contrast enhanced statistics"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local motion vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP tuning metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1050a75fb7ef..ef51911fcfe4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,16 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10	v4l2_fourcc('M', 'T', 'B', 'A') /* Packed bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12	v4l2_fourcc('M', 'T', 'B', 'C') /* Packed bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14	v4l2_fourcc('M', 'T', 'B', 'E') /* Packed bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10	v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12	v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14	v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -750,6 +760,13 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
 
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS	v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A		v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF		v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
 
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v3 4/9] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/pixfmt-mtb8.rst |  49 +++++++++
 Documentation/media/uapi/v4l/pixfmt-mtba.rst |  62 +++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbc.rst |  58 ++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtbe.rst |  70 ++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtf8.rst |  75 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfa.rst |  87 +++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfc.rst | 107 +++++++++++++++++++
 Documentation/media/uapi/v4l/pixfmt-mtfe.rst | 107 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c         |  13 +++
 include/uapi/linux/videodev2.h               |  17 +++
 10 files changed, 645 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtb8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtba.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtbe.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtf8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfa.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfc.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtfe.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtb8.rst b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
new file mode 100644
index 000000000000..2337ccd66277
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtb8.rst
@@ -0,0 +1,49 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B8 ('MTB8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other in memory, with a depth of 8 bits per pixel.
+Each sample is stored in a byte.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtba.rst b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
new file mode 100644
index 000000000000..ade51d5472b0
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtba.rst
@@ -0,0 +1,62 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B10 ('MTBA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbc.rst b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
new file mode 100644
index 000000000000..b122600fddb5
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbc.rst
@@ -0,0 +1,58 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B12 ('MTBC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtbe.rst b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
new file mode 100644
index 000000000000..4b9bc9a62504
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtbe.rst
@@ -0,0 +1,70 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_B14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_B14 ('MTBE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format, meaning all the data for a pixel lie
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtf8.rst b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
new file mode 100644
index 000000000000..51c9ddc4e20d
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtf8.rst
@@ -0,0 +1,75 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F8 ('MTF8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 8 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfa.rst b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
new file mode 100644
index 000000000000..68421c44f5e7
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfa.rst
@@ -0,0 +1,87 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F10 ('MTFA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 10 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfc.rst b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
new file mode 100644
index 000000000000..a3535f5435fa
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfc.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F12 ('MTFC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 12 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtfe.rst b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
new file mode 100644
index 000000000000..324a258e897f
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtfe.rst
@@ -0,0 +1,107 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-MTISP_F14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_F14 ('MTFE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+
+Description
+===========
+
+The four pixel formats are used by Mediatek ISP.
+This is a packed format with a depth of 14 bits per pixel.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+Below is an example of conventional RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index ac87c3e37280..2f536fedd9c4 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1297,6 +1297,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_B8:	descr = "8-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F8:	descr = "8-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B10:	descr = "10-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F10:	descr = "10-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B12:	descr = "12-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F12:	descr = "12-bit Packed Full-G Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_B14:	descr = "14-bit Packed Bayer format"; break;
+	case V4L2_PIX_FMT_MTISP_F14:	descr = "14-bit Packed Full-G Bayer format"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1312,6 +1320,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local contrast enhanced statistics"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local motion vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP tuning metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1050a75fb7ef..ef51911fcfe4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,16 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_B8	v4l2_fourcc('M', 'T', 'B', '8') /* Packed bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_B10	v4l2_fourcc('M', 'T', 'B', 'A') /* Packed bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_B12	v4l2_fourcc('M', 'T', 'B', 'C') /* Packed bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_B14	v4l2_fourcc('M', 'T', 'B', 'E') /* Packed bayer format, 14-bit */
+#define V4L2_PIX_FMT_MTISP_F8	v4l2_fourcc('M', 'T', 'F', '8') /* Full-G bayer format,  8-bit */
+#define V4L2_PIX_FMT_MTISP_F10	v4l2_fourcc('M', 'T', 'F', 'A') /* Full-G bayer format, 10-bit */
+#define V4L2_PIX_FMT_MTISP_F12	v4l2_fourcc('M', 'T', 'F', 'C') /* Full-G bayer format, 12-bit */
+#define V4L2_PIX_FMT_MTISP_F14	v4l2_fourcc('M', 'T', 'F', 'E') /* Full-G bayer format, 14-bit */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -750,6 +760,13 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
 
+/* Vendor specific - Mediatek ISP parameters for firmware */
+#define V4L2_META_FMT_MTISP_PARAMS	v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_3A		v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF		v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS	v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV	v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
 
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 V4L2 control number with 16.
Moreover, add two V4L2 controls for ISP P1 user space
usage.

1. V4L2_CID_MTK_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_MTK_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default value is 0 which outputs the pure raw bayer image
from sesnor, without image processing in ISP HW.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/Makefile       |   3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |   5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   5 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 138 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  38 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 6 files changed, 193 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/Makefile b/drivers/media/platform/mtk-isp/Makefile
new file mode 100644
index 000000000000..c17fb3fc3340
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-y += isp_50/
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..8498fe70e418
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ifeq ($(CONFIG_VIDEO_MEDIATEK_ISP_PASS1),y)
+obj-y += cam/
+endif
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53fb69d3add6
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam-ctrl.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..31d801c82495
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct v4l2_format *fmt;
+
+	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
+		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
+
+	if (is_width)
+		ctrl->val = fmt->fmt.pix_mp.width;
+	else
+		ctrl->val = fmt->fmt.pix_mp.height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
+
+	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
+		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
+	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		handle_ctrl_get_process_raw(ctrl);
+		break;
+	case V4L2_CID_MTK_GET_BIN_WIDTH:
+		handle_ctrl_get_bin_info(ctrl, 1);
+		break;
+	case V4L2_CID_MTK_GET_BIN_HEIGTH:
+		handle_ctrl_get_bin_info(ctrl, 0);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		return handle_ctrl_set_process_raw(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_PROCESSING_RAW,
+	.name = "MTK CAM PROCESSING RAW",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
+	.name = "MTK CAM GET BIN WIDTH",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_WIDTH,
+	.max = IMG_MAX_WIDTH,
+	.step = 1,
+	.def = IMG_MAX_WIDTH,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
+	.name = "MTK CAM GET BIN HEIGHT",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_HEIGHT,
+	.max = IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..0f9349ae0b07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/* The base for the MTK Camera ISP P1 driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_MTK_CAM_BASE			V4L2_CID_USER_MTK_CAM_BASE
+
+/* Control MTK ISP P1 main stream to process raw image data or not.
+ * The default value is 0 which outputs the pure raw bayer data from sensor,
+ * without image processing in ISP HW.
+ */
+#define V4L2_CID_MTK_PROCESSING_RAW		(V4L2_CID_MTK_CAM_BASE + 1)
+
+/* MTK ISP P1 HW supports frontal binning function.
+ * If this function is enabled, the 3A algo. may get the new image resolution
+ * which is binned by ISP P1. If this function is disabled or no supported,
+ * the image resolution will be equal to configured image format.
+ * For this control, it is read only.
+ */
+#define V4L2_CID_MTK_GET_BIN_WIDTH		(V4L2_CID_MTK_CAM_BASE + 2)
+#define V4L2_CID_MTK_GET_BIN_HEIGTH		(V4L2_CID_MTK_CAM_BASE + 3)
+
+#define V4L2_CID_MTK_CAM_MAX			16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 37807f23231e..2db99716f40d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

Reserved Mediatek ISP P1 V4L2 control number with 16.
Moreover, add two V4L2 controls for ISP P1 user space
usage.

1. V4L2_CID_MTK_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_MTK_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default value is 0 which outputs the pure raw bayer image
from sesnor, without image processing in ISP HW.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/Makefile       |   3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |   5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   5 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 138 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  38 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 6 files changed, 193 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/Makefile b/drivers/media/platform/mtk-isp/Makefile
new file mode 100644
index 000000000000..c17fb3fc3340
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-y += isp_50/
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..8498fe70e418
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ifeq ($(CONFIG_VIDEO_MEDIATEK_ISP_PASS1),y)
+obj-y += cam/
+endif
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53fb69d3add6
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam-ctrl.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..31d801c82495
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct v4l2_format *fmt;
+
+	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
+		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
+
+	if (is_width)
+		ctrl->val = fmt->fmt.pix_mp.width;
+	else
+		ctrl->val = fmt->fmt.pix_mp.height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
+
+	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
+		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
+	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		handle_ctrl_get_process_raw(ctrl);
+		break;
+	case V4L2_CID_MTK_GET_BIN_WIDTH:
+		handle_ctrl_get_bin_info(ctrl, 1);
+		break;
+	case V4L2_CID_MTK_GET_BIN_HEIGTH:
+		handle_ctrl_get_bin_info(ctrl, 0);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		return handle_ctrl_set_process_raw(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_PROCESSING_RAW,
+	.name = "MTK CAM PROCESSING RAW",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
+	.name = "MTK CAM GET BIN WIDTH",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_WIDTH,
+	.max = IMG_MAX_WIDTH,
+	.step = 1,
+	.def = IMG_MAX_WIDTH,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
+	.name = "MTK CAM GET BIN HEIGHT",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_HEIGHT,
+	.max = IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..0f9349ae0b07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/* The base for the MTK Camera ISP P1 driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_MTK_CAM_BASE			V4L2_CID_USER_MTK_CAM_BASE
+
+/* Control MTK ISP P1 main stream to process raw image data or not.
+ * The default value is 0 which outputs the pure raw bayer data from sensor,
+ * without image processing in ISP HW.
+ */
+#define V4L2_CID_MTK_PROCESSING_RAW		(V4L2_CID_MTK_CAM_BASE + 1)
+
+/* MTK ISP P1 HW supports frontal binning function.
+ * If this function is enabled, the 3A algo. may get the new image resolution
+ * which is binned by ISP P1. If this function is disabled or no supported,
+ * the image resolution will be equal to configured image format.
+ * For this control, it is read only.
+ */
+#define V4L2_CID_MTK_GET_BIN_WIDTH		(V4L2_CID_MTK_CAM_BASE + 2)
+#define V4L2_CID_MTK_GET_BIN_HEIGTH		(V4L2_CID_MTK_CAM_BASE + 3)
+
+#define V4L2_CID_MTK_CAM_MAX			16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 37807f23231e..2db99716f40d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Reserved Mediatek ISP P1 V4L2 control number with 16.
Moreover, add two V4L2 controls for ISP P1 user space
usage.

1. V4L2_CID_MTK_GET_BIN_INFO
- Provide the image output width & height in case
camera binning mode is enabled.

2. V4L2_CID_MTK_RAW_PATH
- Export the path control of the main stream to user space.
One is pure raw and the other is processing raw.
The default value is 0 which outputs the pure raw bayer image
from sesnor, without image processing in ISP HW.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 drivers/media/platform/mtk-isp/Makefile       |   3 +
 .../media/platform/mtk-isp/isp_50/Makefile    |   5 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |   5 +
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.c         | 138 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-ctrl.h         |  38 +++++
 include/uapi/linux/v4l2-controls.h            |   4 +
 6 files changed, 193 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h

diff --git a/drivers/media/platform/mtk-isp/Makefile b/drivers/media/platform/mtk-isp/Makefile
new file mode 100644
index 000000000000..c17fb3fc3340
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-y += isp_50/
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..8498fe70e418
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ifeq ($(CONFIG_VIDEO_MEDIATEK_ISP_PASS1),y)
+obj-y += cam/
+endif
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53fb69d3add6
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam-ctrl.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
new file mode 100644
index 000000000000..31d801c82495
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/platform_device.h>
+
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam.h"
+
+static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct v4l2_format *fmt;
+
+	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+
+	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
+		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
+
+	if (is_width)
+		ctrl->val = fmt->fmt.pix_mp.width;
+	else
+		ctrl->val = fmt->fmt.pix_mp.height;
+
+	return 0;
+}
+
+static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
+
+	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
+
+	return 0;
+}
+
+static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_cam_dev *cam_dev = ctrl->priv;
+	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
+
+	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
+		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
+	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
+	return 0;
+}
+
+static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		handle_ctrl_get_process_raw(ctrl);
+		break;
+	case V4L2_CID_MTK_GET_BIN_WIDTH:
+		handle_ctrl_get_bin_info(ctrl, 1);
+		break;
+	case V4L2_CID_MTK_GET_BIN_HEIGTH:
+		handle_ctrl_get_bin_info(ctrl, 0);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_MTK_PROCESSING_RAW:
+		return handle_ctrl_set_process_raw(ctrl);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
+	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
+	.s_ctrl = mtk_cam_dev_s_ctrl,
+};
+
+struct v4l2_ctrl_config mtk_cam_controls[] = {
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_PROCESSING_RAW,
+	.name = "MTK CAM PROCESSING RAW",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = 0,
+	.max = 1,
+	.step = 1,
+	.def = 1,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
+	.name = "MTK CAM GET BIN WIDTH",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_WIDTH,
+	.max = IMG_MAX_WIDTH,
+	.step = 1,
+	.def = IMG_MAX_WIDTH,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+	{
+	.ops = &mtk_cam_dev_ctrl_ops,
+	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
+	.name = "MTK CAM GET BIN HEIGHT",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = IMG_MIN_HEIGHT,
+	.max = IMG_MAX_HEIGHT,
+	.step = 1,
+	.def = IMG_MAX_HEIGHT,
+	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
+	},
+};
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl)
+{
+	unsigned int i;
+
+	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
+	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
+	if (hdl->error) {
+		v4l2_ctrl_handler_free(hdl);
+		return hdl->error;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
+		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
new file mode 100644
index 000000000000..0f9349ae0b07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_CTRL_H__
+#define __MTK_CAM_CTRL_H__
+
+#include <media/v4l2-ctrls.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/* The base for the MTK Camera ISP P1 driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_MTK_CAM_BASE			V4L2_CID_USER_MTK_CAM_BASE
+
+/* Control MTK ISP P1 main stream to process raw image data or not.
+ * The default value is 0 which outputs the pure raw bayer data from sensor,
+ * without image processing in ISP HW.
+ */
+#define V4L2_CID_MTK_PROCESSING_RAW		(V4L2_CID_MTK_CAM_BASE + 1)
+
+/* MTK ISP P1 HW supports frontal binning function.
+ * If this function is enabled, the 3A algo. may get the new image resolution
+ * which is binned by ISP P1. If this function is disabled or no supported,
+ * the image resolution will be equal to configured image format.
+ * For this control, it is read only.
+ */
+#define V4L2_CID_MTK_GET_BIN_WIDTH		(V4L2_CID_MTK_CAM_BASE + 2)
+#define V4L2_CID_MTK_GET_BIN_HEIGTH		(V4L2_CID_MTK_CAM_BASE + 3)
+
+#define V4L2_CID_MTK_CAM_MAX			16
+
+int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
+		      struct v4l2_ctrl_handler *hdl);
+
+#endif /* __MTK_CAM_CTRL_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 37807f23231e..2db99716f40d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the mediatek ISP Pass 1 driver controls */
+/* We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and seven video devices during
initialization. Moreover, it also connects with sensor and
seninf drivers with V4L2 async APIs.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 3 files changed, 1848 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 53fb69d3add6..7558593e63f0 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
+mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..117398ed29d2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1674 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * MTK_CAM-v4l2-util is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_CIO_PAD_SRC			4
+#define MTK_CAM_CIO_PAD_SINK			11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_mdev_to_dev(struct media_device *__mdev)
+{
+	return container_of(__mdev,
+		struct mtk_cam_dev, media_dev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req,
+		struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
+				      struct media_request *new_req)
+{
+	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
+
+	if (!cam_dev->streaming) {
+		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
+		spin_lock(&cam_dev->req_lock);
+		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
+		spin_unlock(&cam_dev->req_lock);
+		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
+		return;
+	}
+
+	/* Normal enqueue flow */
+	if (new_req) {
+		mtk_isp_req_enqueue(dev, new_req);
+		return;
+	}
+
+	/* Flush all media requests wehen first stream on */
+	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
+		list_del(&req->list);
+		mtk_isp_req_enqueue(dev, &req->req);
+	}
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
+
+	vb2_request_queue(req);
+	mtk_cam_req_try_isp_queue(cam_dev, req);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
+					__u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+
+	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
+				       __u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+
+	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
+		stride = ALIGN(stride, 4);
+
+	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Calcuate mplane pix format */
+static void
+mtk_cam_dev_cal_mplane_fmt(struct device *dev,
+			   struct v4l2_pix_format_mplane *dest_fmt,
+			   unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = img_cal_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+static void
+mtk_cam_dev_set_img_fmt(struct device *dev,
+			struct v4l2_pix_format_mplane *dest_fmt,
+			const struct v4l2_pix_format_mplane *src_fmt,
+			unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct device *dev,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_set_img_fmt(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+static int mtk_cam_isp_open(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	int ret;
+
+	mutex_lock(&cam_dev->lock);
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto unlock;
+
+	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
+	if (ret)
+		dev_err(dev, "%s fail:%d", __func__, ret);
+
+unlock:
+	mutex_unlock(&cam_dev->lock);
+
+	return ret;
+}
+
+static int mtk_cam_isp_release(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	mutex_lock(&cam_dev->lock);
+	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
+	vb2_fop_release(file);
+	mutex_unlock(&cam_dev->lock);
+
+	return 0;
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "Sensor is not connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
+	if (!cam_dev->sensor)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_req_try_isp_queue(cam_dev, NULL);
+	isp_composer_stream(dev, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	isp_composer_stream(dev, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
+
+	return on ? mtk_isp_power_init(cam_dev) :
+		    mtk_isp_power_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam_dev->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		buf->vbb.request_fd,
+		buf->vbb.vb2_buf.index);
+
+	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
+	if (vb->vb2_queue->uses_requests)
+		return;
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *smem_dev = cam_dev->smem_dev;
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	/* scp address is only valid for meta input buffer */
+	if (node->desc.smem_alloc)
+		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
+							      buf->daddr);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size)
+		return -EINVAL;
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct mtk_cam_dev_request *req, *req0;
+	struct media_request_object *obj, *obj0;
+	struct vb2_buffer *vb;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (vb->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(vb, state);
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	spin_lock(&cam_dev->req_lock);
+	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
+		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
+			vb = container_of(obj, struct vb2_buffer, req_obj);
+			if (vb->state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(vb, state);
+		}
+		list_del(&req->list);
+	}
+	spin_unlock(&cam_dev->req_lock);
+
+	if (node->vbq.uses_requests)
+		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int node_count = cam_dev->subdev.entity.use_count;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enable\n", node->id);
+		ret = -ENOLINK;
+		goto fail_no_link;
+	}
+
+	dev_dbg(dev, "%s: count info:%d:%d", __func__,
+		atomic_read(&cam_dev->streamed_node_count), node_count);
+
+	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret) {
+		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
+		goto fail_start_pipeline;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
+		goto fail_stream_on;
+	}
+
+	return 0;
+
+fail_stream_on:
+	media_pipeline_stop(&node->vdev.entity);
+fail_start_pipeline:
+	atomic_dec(&cam_dev->streamed_node_count);
+fail_no_link:
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	if (!node->enabled)
+		return;
+
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+
+	dev_dbg(dev, "%s: count info:%d", __func__,
+		cam_dev->subdev.entity.stream_count);
+
+	/* Check the first node to stream-off */
+	if (!cam_dev->subdev.entity.stream_count)
+		return;
+
+	media_pipeline_stop(&node->vdev.entity);
+
+	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+		dev_err(dev, "failed to stop streaming\n");
+}
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (!node->desc.num_fmts)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	const struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc, in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
+				   &in_fmt->fmt.pix_mp, node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+				     struct v4l2_input *input)
+{
+	if (input->index)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
+				  unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_input(struct file *file,
+				  void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+	.s_power = mtk_cam_sd_s_power,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = mtk_cam_isp_open,
+	.release = mtk_cam_isp_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+
+	return media_device_register(media_dev);
+}
+
+static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
+	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
+	int ret;
+
+	cam_dev->subdev_pads[i].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->enabled = false;
+	node->id = i;
+	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     &node->desc,
+				     &node->vdev_fmt);
+
+	/* Initialize vbq */
+	vbq->type = node->vdev_fmt.type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam_dev->smem_dev;
+	} else {
+		vbq->dev = &cam_dev->pdev->dev;
+	}
+
+	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		vdev->entity.function =
+			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+	/* Put the process hub sub device in the vb2 private data */
+	vbq->drv_priv = cam_dev;
+	vbq->lock = &node->lock;
+	vbq->supports_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_vb2_queue;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 MTK_CAM_DEV_P1_NAME, node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->lock;
+	vdev->v4l2_dev = &cam_dev->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.ops = NULL;
+	/* Enable private control for image video devices */
+	if (node->desc.image) {
+		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+		vdev->ctrl_handler = &node->ctrl_handler;
+	}
+	video_set_drvdata(vdev, cam_dev);
+	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vdev;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam_dev->subdev.entity,
+					    i, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam_dev->subdev.entity,
+					    i, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_link;
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+	INIT_LIST_HEAD(&node->pending_list);
+
+	return 0;
+
+fail_link:
+	video_unregister_device(vdev);
+fail_vdev:
+	vb2_queue_release(vbq);
+fail_vb2_queue:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
+
+	/* Set up v4l2 device */
+	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
+	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		ret = mtk_cam_video_register_device(cam_dev, i);
+		if (ret)
+			goto fail_video_register;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_video_register:
+	i--;
+	for (; i >= 0; i--) {
+		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
+		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
+		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		dev = &cam_dev->vdev_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_CIO_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_CIO_PAD_SINK,
+				    0);
+	if (ret) {
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = NULL;
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
+		NULL);
+	if (ret)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = IMG_MAX_WIDTH,
+			.min_width = IMG_MIN_WIDTH,
+			.max_height = IMG_MAX_HEIGHT,
+			.min_height = IMG_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = RRZ_MAX_WIDTH,
+			.min_width = RRZ_MIN_WIDTH,
+			.max_height = RRZ_MAX_HEIGHT,
+			.min_height = RRZ_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[0],
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[1],
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+	mtk_cam_dev_queue_setup(cam_dev);
+	/* v4l2 sub-device registration */
+
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		mtk_cam_v4l2_unregister(cam_dev);
+		return ret;
+	}
+
+	spin_lock_init(&cam_dev->req_lock);
+	INIT_LIST_HEAD(&cam_dev->req_list);
+	mutex_init(&cam_dev->lock);
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	mutex_destroy(&cam_dev->lock);
+
+	return 0;
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_P1_META_IN_0			0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT		1
+#define MTK_CAM_P1_PACKED_BIN_OUT		2
+#define MTK_CAM_P1_META_OUT_0			3
+#define MTK_CAM_P1_META_OUT_1			4
+#define MTK_CAM_P1_META_OUT_2			5
+#define MTK_CAM_P1_META_OUT_3			6
+#define MTK_CAM_P1_TOTAL_CAPTURE		6
+
+#define MTK_CAM_P1_TOTAL_NODES			7
+
+struct mtk_cam_dev_request {
+	struct media_request	req;
+	struct list_head	list;
+};
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+	unsigned int		node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ * @frmsizes:	supported frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct mtk_cam_dev_node_desc desc;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @lock:       The mutex protecting video device open/release operations
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @streamed_node_count: The number of V4L2 video device nodes are streaming on
+ * @req_list:	Lins to keep media requests before streaming on
+ * @req_lock:	Protect the req_list data
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	/* protect video device open/release operations */
+	struct mutex lock;
+	unsigned int streaming:1;
+	atomic_t streamed_node_count;
+	struct list_head req_list;
+	/* protect for req_list */
+	spinlock_t req_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and seven video devices during
initialization. Moreover, it also connects with sensor and
seninf drivers with V4L2 async APIs.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 3 files changed, 1848 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 53fb69d3add6..7558593e63f0 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
+mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..117398ed29d2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1674 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * MTK_CAM-v4l2-util is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_CIO_PAD_SRC			4
+#define MTK_CAM_CIO_PAD_SINK			11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_mdev_to_dev(struct media_device *__mdev)
+{
+	return container_of(__mdev,
+		struct mtk_cam_dev, media_dev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req,
+		struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
+				      struct media_request *new_req)
+{
+	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
+
+	if (!cam_dev->streaming) {
+		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
+		spin_lock(&cam_dev->req_lock);
+		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
+		spin_unlock(&cam_dev->req_lock);
+		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
+		return;
+	}
+
+	/* Normal enqueue flow */
+	if (new_req) {
+		mtk_isp_req_enqueue(dev, new_req);
+		return;
+	}
+
+	/* Flush all media requests wehen first stream on */
+	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
+		list_del(&req->list);
+		mtk_isp_req_enqueue(dev, &req->req);
+	}
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
+
+	vb2_request_queue(req);
+	mtk_cam_req_try_isp_queue(cam_dev, req);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
+					__u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+
+	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
+				       __u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+
+	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
+		stride = ALIGN(stride, 4);
+
+	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Calcuate mplane pix format */
+static void
+mtk_cam_dev_cal_mplane_fmt(struct device *dev,
+			   struct v4l2_pix_format_mplane *dest_fmt,
+			   unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = img_cal_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+static void
+mtk_cam_dev_set_img_fmt(struct device *dev,
+			struct v4l2_pix_format_mplane *dest_fmt,
+			const struct v4l2_pix_format_mplane *src_fmt,
+			unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct device *dev,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_set_img_fmt(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+static int mtk_cam_isp_open(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	int ret;
+
+	mutex_lock(&cam_dev->lock);
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto unlock;
+
+	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
+	if (ret)
+		dev_err(dev, "%s fail:%d", __func__, ret);
+
+unlock:
+	mutex_unlock(&cam_dev->lock);
+
+	return ret;
+}
+
+static int mtk_cam_isp_release(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	mutex_lock(&cam_dev->lock);
+	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
+	vb2_fop_release(file);
+	mutex_unlock(&cam_dev->lock);
+
+	return 0;
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "Sensor is not connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
+	if (!cam_dev->sensor)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_req_try_isp_queue(cam_dev, NULL);
+	isp_composer_stream(dev, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	isp_composer_stream(dev, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
+
+	return on ? mtk_isp_power_init(cam_dev) :
+		    mtk_isp_power_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam_dev->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		buf->vbb.request_fd,
+		buf->vbb.vb2_buf.index);
+
+	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
+	if (vb->vb2_queue->uses_requests)
+		return;
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *smem_dev = cam_dev->smem_dev;
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	/* scp address is only valid for meta input buffer */
+	if (node->desc.smem_alloc)
+		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
+							      buf->daddr);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size)
+		return -EINVAL;
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct mtk_cam_dev_request *req, *req0;
+	struct media_request_object *obj, *obj0;
+	struct vb2_buffer *vb;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (vb->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(vb, state);
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	spin_lock(&cam_dev->req_lock);
+	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
+		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
+			vb = container_of(obj, struct vb2_buffer, req_obj);
+			if (vb->state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(vb, state);
+		}
+		list_del(&req->list);
+	}
+	spin_unlock(&cam_dev->req_lock);
+
+	if (node->vbq.uses_requests)
+		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int node_count = cam_dev->subdev.entity.use_count;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enable\n", node->id);
+		ret = -ENOLINK;
+		goto fail_no_link;
+	}
+
+	dev_dbg(dev, "%s: count info:%d:%d", __func__,
+		atomic_read(&cam_dev->streamed_node_count), node_count);
+
+	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret) {
+		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
+		goto fail_start_pipeline;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
+		goto fail_stream_on;
+	}
+
+	return 0;
+
+fail_stream_on:
+	media_pipeline_stop(&node->vdev.entity);
+fail_start_pipeline:
+	atomic_dec(&cam_dev->streamed_node_count);
+fail_no_link:
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	if (!node->enabled)
+		return;
+
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+
+	dev_dbg(dev, "%s: count info:%d", __func__,
+		cam_dev->subdev.entity.stream_count);
+
+	/* Check the first node to stream-off */
+	if (!cam_dev->subdev.entity.stream_count)
+		return;
+
+	media_pipeline_stop(&node->vdev.entity);
+
+	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+		dev_err(dev, "failed to stop streaming\n");
+}
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (!node->desc.num_fmts)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	const struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc, in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
+				   &in_fmt->fmt.pix_mp, node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+				     struct v4l2_input *input)
+{
+	if (input->index)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
+				  unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_input(struct file *file,
+				  void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+	.s_power = mtk_cam_sd_s_power,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = mtk_cam_isp_open,
+	.release = mtk_cam_isp_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+
+	return media_device_register(media_dev);
+}
+
+static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
+	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
+	int ret;
+
+	cam_dev->subdev_pads[i].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->enabled = false;
+	node->id = i;
+	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     &node->desc,
+				     &node->vdev_fmt);
+
+	/* Initialize vbq */
+	vbq->type = node->vdev_fmt.type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam_dev->smem_dev;
+	} else {
+		vbq->dev = &cam_dev->pdev->dev;
+	}
+
+	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		vdev->entity.function =
+			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+	/* Put the process hub sub device in the vb2 private data */
+	vbq->drv_priv = cam_dev;
+	vbq->lock = &node->lock;
+	vbq->supports_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_vb2_queue;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 MTK_CAM_DEV_P1_NAME, node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->lock;
+	vdev->v4l2_dev = &cam_dev->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.ops = NULL;
+	/* Enable private control for image video devices */
+	if (node->desc.image) {
+		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+		vdev->ctrl_handler = &node->ctrl_handler;
+	}
+	video_set_drvdata(vdev, cam_dev);
+	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vdev;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam_dev->subdev.entity,
+					    i, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam_dev->subdev.entity,
+					    i, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_link;
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+	INIT_LIST_HEAD(&node->pending_list);
+
+	return 0;
+
+fail_link:
+	video_unregister_device(vdev);
+fail_vdev:
+	vb2_queue_release(vbq);
+fail_vb2_queue:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
+
+	/* Set up v4l2 device */
+	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
+	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		ret = mtk_cam_video_register_device(cam_dev, i);
+		if (ret)
+			goto fail_video_register;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_video_register:
+	i--;
+	for (; i >= 0; i--) {
+		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
+		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
+		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		dev = &cam_dev->vdev_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_CIO_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_CIO_PAD_SINK,
+				    0);
+	if (ret) {
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = NULL;
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
+		NULL);
+	if (ret)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = IMG_MAX_WIDTH,
+			.min_width = IMG_MIN_WIDTH,
+			.max_height = IMG_MAX_HEIGHT,
+			.min_height = IMG_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = RRZ_MAX_WIDTH,
+			.min_width = RRZ_MIN_WIDTH,
+			.max_height = RRZ_MAX_HEIGHT,
+			.min_height = RRZ_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[0],
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[1],
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+	mtk_cam_dev_queue_setup(cam_dev);
+	/* v4l2 sub-device registration */
+
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		mtk_cam_v4l2_unregister(cam_dev);
+		return ret;
+	}
+
+	spin_lock_init(&cam_dev->req_lock);
+	INIT_LIST_HEAD(&cam_dev->req_list);
+	mutex_init(&cam_dev->lock);
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	mutex_destroy(&cam_dev->lock);
+
+	return 0;
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_P1_META_IN_0			0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT		1
+#define MTK_CAM_P1_PACKED_BIN_OUT		2
+#define MTK_CAM_P1_META_OUT_0			3
+#define MTK_CAM_P1_META_OUT_1			4
+#define MTK_CAM_P1_META_OUT_2			5
+#define MTK_CAM_P1_META_OUT_3			6
+#define MTK_CAM_P1_TOTAL_CAPTURE		6
+
+#define MTK_CAM_P1_TOTAL_NODES			7
+
+struct mtk_cam_dev_request {
+	struct media_request	req;
+	struct list_head	list;
+};
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+	unsigned int		node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ * @frmsizes:	supported frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct mtk_cam_dev_node_desc desc;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @lock:       The mutex protecting video device open/release operations
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @streamed_node_count: The number of V4L2 video device nodes are streaming on
+ * @req_list:	Lins to keep media requests before streaming on
+ * @req_lock:	Protect the req_list data
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	/* protect video device open/release operations */
+	struct mutex lock;
+	unsigned int streaming:1;
+	atomic_t streamed_node_count;
+	struct list_head req_list;
+	/* protect for req_list */
+	spinlock_t req_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Implement standard V4L2 video driver that utilizes V4L2
and media framework APIs. In this driver, supports one media
device, one sub-device and seven video devices during
initialization. Moreover, it also connects with sensor and
seninf drivers with V4L2 async APIs.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "media: support Mediatek sensor interface driver"[1].

ISP P1 sub-device communicates with seninf sub-device with CIO.

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/10979135/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
 3 files changed, 1848 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 53fb69d3add6..7558593e63f0 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
+mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
new file mode 100644
index 000000000000..117398ed29d2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
@@ -0,0 +1,1674 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Mediatek Corporation.
+ * Copyright (c) 2017 Intel Corporation.
+ *
+ * MTK_CAM-v4l2-util is highly based on Intel IPU3 ImgU driver.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-smem.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define MTK_CAM_CIO_PAD_SRC			4
+#define MTK_CAM_CIO_PAD_SINK			11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_subdev_to_dev(struct v4l2_subdev *__sd)
+{
+	return container_of(__sd,
+		struct mtk_cam_dev, subdev);
+}
+
+static inline struct mtk_cam_dev *
+mtk_cam_mdev_to_dev(struct media_device *__mdev)
+{
+	return container_of(__mdev,
+		struct mtk_cam_dev, media_dev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq,
+		struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req,
+		struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb,
+		struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
+				      struct media_request *new_req)
+{
+	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
+	struct device *dev = &cam_dev->pdev->dev;
+
+	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
+
+	if (!cam_dev->streaming) {
+		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
+		spin_lock(&cam_dev->req_lock);
+		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
+		spin_unlock(&cam_dev->req_lock);
+		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
+		return;
+	}
+
+	/* Normal enqueue flow */
+	if (new_req) {
+		mtk_isp_req_enqueue(dev, new_req);
+		return;
+	}
+
+	/* Flush all media requests wehen first stream on */
+	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
+		list_del(&req->list);
+		mtk_isp_req_enqueue(dev, &req->req);
+	}
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
+
+	vb2_request_queue(req);
+	mtk_cam_req_try_isp_queue(cam_dev, req);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
+					__u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
+
+	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
+				       __u32 pix_fmt)
+{
+	__u32 stride;
+	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
+
+	width = ALIGN(width, 4);
+	stride = DIV_ROUND_UP(width * 3, 2);
+	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
+
+	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
+		stride = ALIGN(stride, 4);
+
+	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
+
+	return stride;
+}
+
+static __u32 img_cal_stride(struct device *dev,
+			    int node_id,
+			    __u32 width,
+			    __u32 pix_fmt)
+{
+	__u32 bpl;
+
+	/* Currently, only support one_pixel_mode */
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
+		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
+	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
+		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
+
+	/* For DIP HW constrained, it needs 4 byte alignment */
+	bpl = ALIGN(bpl, 4);
+
+	return bpl;
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	unsigned int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Calcuate mplane pix format */
+static void
+mtk_cam_dev_cal_mplane_fmt(struct device *dev,
+			   struct v4l2_pix_format_mplane *dest_fmt,
+			   unsigned int node_id)
+{
+	unsigned int i;
+	__u32 bpl, sizeimage, imagsize;
+
+	imagsize = 0;
+	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
+		bpl = img_cal_stride(dev,
+				     node_id,
+				     dest_fmt->width,
+				     dest_fmt->pixelformat);
+		sizeimage = bpl * dest_fmt->height;
+		imagsize += sizeimage;
+		dest_fmt->plane_fmt[i].bytesperline = bpl;
+		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
+		memset(dest_fmt->plane_fmt[i].reserved,
+		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
+		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
+			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
+	}
+
+	if (dest_fmt->num_planes == 1)
+		dest_fmt->plane_fmt[0].sizeimage = imagsize;
+}
+
+static void
+mtk_cam_dev_set_img_fmt(struct device *dev,
+			struct v4l2_pix_format_mplane *dest_fmt,
+			const struct v4l2_pix_format_mplane *src_fmt,
+			unsigned int node_id)
+{
+	dest_fmt->width = src_fmt->width;
+	dest_fmt->height = src_fmt->height;
+	dest_fmt->pixelformat = src_fmt->pixelformat;
+	dest_fmt->field = src_fmt->field;
+	dest_fmt->colorspace = src_fmt->colorspace;
+	dest_fmt->num_planes = src_fmt->num_planes;
+	/* Use default */
+	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest_fmt->xfer_func =
+		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
+	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
+
+	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
+		__func__,
+		(dest_fmt->pixelformat & 0xFF),
+		(dest_fmt->pixelformat >> 8) & 0xFF,
+		(dest_fmt->pixelformat >> 16) & 0xFF,
+		(dest_fmt->pixelformat >> 24) & 0xFF,
+		dest_fmt->width,
+		dest_fmt->height);
+
+	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct device *dev,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (queue_desc->image) {
+		mtk_cam_dev_set_img_fmt(dev,
+					&dest->fmt.pix_mp,
+					&default_fmt->fmt.pix_mp,
+					queue_desc->id);
+	} else {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+	}
+}
+
+static int mtk_cam_isp_open(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	int ret;
+
+	mutex_lock(&cam_dev->lock);
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto unlock;
+
+	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
+	if (ret)
+		dev_err(dev, "%s fail:%d", __func__, ret);
+
+unlock:
+	mutex_unlock(&cam_dev->lock);
+
+	return ret;
+}
+
+static int mtk_cam_isp_release(struct file *file)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	mutex_lock(&cam_dev->lock);
+	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
+	vb2_fop_release(file);
+	mutex_unlock(&cam_dev->lock);
+
+	return 0;
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
+{
+	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
+	struct media_entity *entity;
+	struct device *dev = &cam_dev->pdev->dev;
+	struct v4l2_subdev *sensor;
+
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x\n",
+			entity->name, entity->function);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "Sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "Sensor is not connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	/* Align vb2_core_streamon design */
+	if (cam_dev->streaming) {
+		dev_warn(dev, "already streaming\n", dev);
+		return 0;
+	}
+
+	if (!cam_dev->seninf) {
+		dev_err(dev, "no seninf connected:%d\n", ret);
+		return -EPERM;
+	}
+
+	/* Get active sensor from graph topology */
+	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
+	if (!cam_dev->sensor)
+		return -EPERM;
+
+	ret = mtk_isp_config(dev);
+	if (ret)
+		return -EPERM;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "%s stream on failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		goto fail_sensor_on;
+	}
+
+	cam_dev->streaming = true;
+	mtk_cam_req_try_isp_queue(cam_dev, NULL);
+	isp_composer_stream(dev, 1);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_on:
+	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+
+	return -EPERM;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	if (!cam_dev->streaming) {
+		dev_warn(dev, "already stream off");
+		return 0;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "%s stream off failed:%d\n",
+			cam_dev->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	isp_composer_stream(dev, 0);
+	cam_dev->streaming = false;
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	if (enable)
+		return mtk_cam_cio_stream_on(cam_dev);
+	else
+		return mtk_cam_cio_stream_off(cam_dev);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
+{
+	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
+
+	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
+
+	return on ? mtk_isp_power_init(cam_dev) :
+		    mtk_isp_power_release(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam_dev->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = &mtk_cam_dev->pdev->dev;
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
+		__func__,
+		node->id,
+		buf->vbb.request_fd,
+		buf->vbb.vb2_buf.index);
+
+	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
+	if (vb->vb2_queue->uses_requests)
+		return;
+
+	/* Added the buffer into the tracking list */
+	spin_lock(&node->slock);
+	list_add_tail(&buf->list, &node->pending_list);
+	spin_unlock(&node->slock);
+
+	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *smem_dev = cam_dev->smem_dev;
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
+	buf->scp_addr = 0;
+
+	/* scp address is only valid for meta input buffer */
+	if (node->desc.smem_alloc)
+		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
+							      buf->daddr);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size)
+		return -EINVAL;
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct mtk_cam_dev_request *req, *req0;
+	struct media_request_object *obj, *obj0;
+	struct vb2_buffer *vb;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
+
+	/* Return all buffers */
+	spin_lock(&node->slock);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (vb->state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(vb, state);
+		list_del(&b->list);
+	}
+	spin_unlock(&node->slock);
+
+	spin_lock(&cam_dev->req_lock);
+	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
+		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
+			vb = container_of(obj, struct vb2_buffer, req_obj);
+			if (vb->state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(vb, state);
+		}
+		list_del(&req->list);
+	}
+	spin_unlock(&cam_dev->req_lock);
+
+	if (node->vbq.uses_requests)
+		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+	unsigned int node_count = cam_dev->subdev.entity.use_count;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enable\n", node->id);
+		ret = -ENOLINK;
+		goto fail_no_link;
+	}
+
+	dev_dbg(dev, "%s: count info:%d:%d", __func__,
+		atomic_read(&cam_dev->streamed_node_count), node_count);
+
+	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
+		return 0;
+
+	/* Start streaming of the whole pipeline now */
+	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
+	if (ret) {
+		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
+		goto fail_start_pipeline;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
+		goto fail_stream_on;
+	}
+
+	return 0;
+
+fail_stream_on:
+	media_pipeline_stop(&node->vdev.entity);
+fail_start_pipeline:
+	atomic_dec(&cam_dev->streamed_node_count);
+fail_no_link:
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = &cam_dev->pdev->dev;
+
+	if (!node->enabled)
+		return;
+
+	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
+
+	dev_dbg(dev, "%s: count info:%d", __func__,
+		cam_dev->subdev.entity.stream_count);
+
+	/* Check the first node to stream-off */
+	if (!cam_dev->subdev.entity.stream_count)
+		return;
+
+	media_pipeline_stop(&node->vdev.entity);
+
+	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
+		dev_err(dev, "failed to stop streaming\n");
+}
+
+static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req,
+				   dev->v4l2_dev.ctrl_handler);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+
+	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
+	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam_dev->media_dev.dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (!node->desc.num_fmts)
+		return -EINVAL;
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *in_fmt)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	const struct v4l2_format *dev_fmt;
+	__u32  width, height;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
+		__func__,
+		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
+		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
+		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
+		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
+
+	width = in_fmt->fmt.pix_mp.width;
+	height = in_fmt->fmt.pix_mp.height;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
+				       in_fmt->fmt.pix_mp.pixelformat);
+	if (dev_fmt) {
+		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+					&in_fmt->fmt.pix_mp,
+					&dev_fmt->fmt.pix_mp,
+					node->id);
+	} else {
+		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+					     &node->desc, in_fmt);
+	}
+	in_fmt->fmt.pix_mp.width = clamp_t(u32,
+					   width,
+					   CAM_MIN_WIDTH,
+					   in_fmt->fmt.pix_mp.width);
+	in_fmt->fmt.pix_mp.height = clamp_t(u32,
+					    height,
+					    CAM_MIN_HEIGHT,
+					    in_fmt->fmt.pix_mp.height);
+	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
+				   &in_fmt->fmt.pix_mp, node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam_dev = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (cam_dev->streaming)
+		return -EBUSY;
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+
+	/* Configure to video device */
+	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
+				&node->vdev_fmt.fmt.pix_mp,
+				&f->fmt.pix_mp,
+				node->id);
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
+				     struct v4l2_input *input)
+{
+	if (input->index)
+		return -EINVAL;
+
+	strscpy(input->name, "camera", sizeof(input->name));
+	input->type = V4L2_INPUT_TYPE_CAMERA;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
+				  unsigned int *input)
+{
+	*input = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_input(struct file *file,
+				  void *fh, unsigned int input)
+{
+	return input == 0 ? 0 : -EINVAL;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	strscpy(f->description, node->desc.description,
+		sizeof(node->desc.description));
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+	.s_power = mtk_cam_sd_s_power,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = mtk_cam_isp_open,
+	.release = mtk_cam_isp_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_req_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct device *dev,
+				  struct media_device *media_dev)
+{
+	media_dev->dev = dev;
+	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_req_ops;
+
+	return media_device_register(media_dev);
+}
+
+static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
+	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
+	int ret;
+
+	cam_dev->subdev_pads[i].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->enabled = false;
+	node->id = i;
+	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
+	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
+				     &node->desc,
+				     &node->vdev_fmt);
+
+	/* Initialize vbq */
+	vbq->type = node->vdev_fmt.type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam_dev->smem_dev;
+	} else {
+		vbq->dev = &cam_dev->pdev->dev;
+	}
+
+	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		vdev->entity.function =
+			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
+	/* Put the process hub sub device in the vb2 private data */
+	vbq->drv_priv = cam_dev;
+	vbq->lock = &node->lock;
+	vbq->supports_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_vb2_queue;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 MTK_CAM_DEV_P1_NAME, node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->lock;
+	vdev->v4l2_dev = &cam_dev->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.ops = NULL;
+	/* Enable private control for image video devices */
+	if (node->desc.image) {
+		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
+		vdev->ctrl_handler = &node->ctrl_handler;
+	}
+	video_set_drvdata(vdev, cam_dev);
+	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vdev;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam_dev->subdev.entity,
+					    i, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam_dev->subdev.entity,
+					    i, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_link;
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->lock);
+	spin_lock_init(&node->slock);
+	INIT_LIST_HEAD(&node->pending_list);
+
+	return 0;
+
+fail_link:
+	video_unregister_device(vdev);
+fail_vdev:
+	vb2_queue_release(vbq);
+fail_vb2_queue:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	/* Total pad numbers is video devices + one seninf pad */
+	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	unsigned int i;
+	int ret;
+
+	ret = mtk_cam_media_register(dev,
+				     &cam_dev->media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+	dev_info(dev, "Register media device: %s, 0x%pK",
+		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
+
+	/* Set up v4l2 device */
+	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
+	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_v4l2_dev;
+	}
+	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
+
+	/* Initialize subdev media entity */
+	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
+					    sizeof(*cam_dev->subdev_pads),
+					    GFP_KERNEL);
+	if (!cam_dev->subdev_pads) {
+		ret = -ENOMEM;
+		goto fail_subdev_pads;
+	}
+
+	ret = media_entity_pads_init(&cam_dev->subdev.entity,
+				     num_subdev_pads,
+				     cam_dev->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed initialize media pads:%d:\n", ret);
+		goto fail_subdev_pads;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_subdev_pads; i++)
+		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
+	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
+	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
+	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
+		 "%s", MTK_CAM_DEV_P1_NAME);
+	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
+
+	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev:%d\n", ret);
+		goto fail_subdev;
+	}
+	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		ret = mtk_cam_video_register_device(cam_dev, i);
+		if (ret)
+			goto fail_video_register;
+	}
+
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_video_register:
+	i--;
+	for (; i >= 0; i--) {
+		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
+		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
+		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
+	}
+fail_subdev:
+	media_entity_cleanup(&cam_dev->subdev.entity);
+fail_subdev_pads:
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+fail_v4l2_dev:
+	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i;
+	struct mtk_cam_video_device *dev;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		dev = &cam_dev->vdev_nodes[i];
+		video_unregister_device(&dev->vdev);
+		media_entity_cleanup(&dev->vdev.entity);
+		mutex_destroy(&dev->lock);
+		if (dev->desc.image)
+			v4l2_ctrl_handler_free(&dev->ctrl_handler);
+	}
+
+	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
+
+	v4l2_device_unregister_subdev(&cam_dev->subdev);
+	media_entity_cleanup(&cam_dev->subdev.entity);
+	kfree(cam_dev->subdev_pads);
+
+	v4l2_device_unregister(&cam_dev->v4l2_dev);
+	media_device_unregister(&cam_dev->media_dev);
+	media_device_cleanup(&cam_dev->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam_dev->seninf->entity,
+				    MTK_CAM_CIO_PAD_SRC,
+				    &cam_dev->subdev.entity,
+				    MTK_CAM_CIO_PAD_SINK,
+				    0);
+	if (ret) {
+		dev_err(dev, "fail to create pad link %s %s err:%d\n",
+			cam_dev->seninf->entity.name,
+			cam_dev->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	dev_info(dev, "Complete the v4l2 registration\n");
+
+	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = sd;
+	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam_dev =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam_dev->seninf = NULL;
+	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	return mtk_cam_dev_complete(notifier);
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	int ret;
+
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
+		NULL);
+	if (ret)
+		return ret;
+
+	if (!cam_dev->notifier.num_subdevs)
+		return -ENODEV;
+
+	cam_dev->notifier.ops = &mtk_cam_async_ops;
+	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
+					   &cam_dev->notifier);
+	if (ret) {
+		dev_err(&cam_dev->pdev->dev,
+			"failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam_dev->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
+{
+	v4l2_async_notifier_unregister(&cam_dev->notifier);
+	v4l2_async_notifier_cleanup(&cam_dev->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
+	.vidioc_g_input = mtk_cam_vidioc_g_input,
+	.vidioc_s_input = mtk_cam_vidioc_s_input,
+	/* buffer queue management */
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 128 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 300 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 160 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 72 * PAGE_SIZE,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = RRZ_MAX_WIDTH,
+			.height = RRZ_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_RAW,
+			.num_planes = 1,
+		},
+	},
+};
+
+static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = IMG_MAX_WIDTH,
+			.min_width = IMG_MIN_WIDTH,
+			.max_height = IMG_MAX_HEIGHT,
+			.min_height = IMG_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+	{
+		.index = 0,
+		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+		.stepwise = {
+			.max_width = RRZ_MAX_WIDTH,
+			.min_width = RRZ_MIN_WIDTH,
+			.max_height = RRZ_MAX_HEIGHT,
+			.min_height = RRZ_MIN_HEIGHT,
+			.step_height = 1,
+			.step_width = 1,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.description = "ISP tuning parameters",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.capture = false,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[0],
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.capture = true,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 1,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &img_frm_size_nums[1],
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.description = "AE/AWB histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.description = "AF histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.description = "Local contrast enhanced statistics",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.description = "Local motion vector histogram",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.capture = true,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.num_fmts = ARRAY_SIZE(meta_fmts),
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int i, node_idx;
+
+	node_idx = 0;
+
+	/* Setup the output queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
+		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev)
+{
+	int ret;
+
+	cam_dev->pdev = pdev;
+	mtk_cam_dev_queue_setup(cam_dev);
+	/* v4l2 sub-device registration */
+
+	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
+		MTK_CAM_DEV_P1_NAME);
+	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam_dev);
+	if (ret) {
+		mtk_cam_v4l2_unregister(cam_dev);
+		return ret;
+	}
+
+	spin_lock_init(&cam_dev->req_lock);
+	INIT_LIST_HEAD(&cam_dev->req_list);
+	mutex_init(&cam_dev->lock);
+
+	return 0;
+}
+
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev)
+{
+	mtk_cam_v4l2_async_unregister(cam_dev);
+	mtk_cam_v4l2_unregister(cam_dev);
+
+	mutex_destroy(&cam_dev->lock);
+
+	return 0;
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_DEV_V4L2_H__
+#define __MTK_CAM_DEV_V4L2_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
+
+#define MTK_CAM_P1_META_IN_0			0
+#define MTK_CAM_P1_TOTAL_OUTPUT		1
+
+#define MTK_CAM_P1_MAIN_STREAM_OUT		1
+#define MTK_CAM_P1_PACKED_BIN_OUT		2
+#define MTK_CAM_P1_META_OUT_0			3
+#define MTK_CAM_P1_META_OUT_1			4
+#define MTK_CAM_P1_META_OUT_2			5
+#define MTK_CAM_P1_META_OUT_3			6
+#define MTK_CAM_P1_TOTAL_CAPTURE		6
+
+#define MTK_CAM_P1_TOTAL_NODES			7
+
+struct mtk_cam_dev_request {
+	struct media_request	req;
+	struct list_head	list;
+};
+
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer	vbb;
+	struct list_head	list;
+	/* Intenal part */
+	dma_addr_t		daddr;
+	dma_addr_t		scp_addr;
+	unsigned int		node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - node attributes
+ *
+ * @id:		 id of the context queue
+ * @name:	 media entity name
+ * @description: descritpion of node
+ * @cap:	 mapped to V4L2 capabilities
+ * @buf_type:	 mapped to V4L2 buffer type
+ * @dma_port:	 the dma port associated to the buffer
+ * @link_flags:	 default media link flags
+ * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
+ * @capture:	 true for capture queue (device to user)
+ *		 false for output queue (from user to device)
+ * @image:	 true for image node, false for meta node
+ * @num_fmts:	 the number of supported formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum V4L2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts:	supported format
+ * @frmsizes:	supported frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	char *name;
+	char *description;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 capture:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure.
+ *
+ * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
+ * @enabled:	Indicate the device is enabled or not
+ * @vdev_fmt:	The V4L2 format of video device
+ * @vdev_apd:	The media pad graph object of video device
+ * @vbq:	A videobuf queue of video device
+ * @desc:	The node attributes of video device
+ * @ctrl_handler:	The control handler of video device
+ * @pending_list:	List for pending buffers before enqueuing into driver
+ * @lock:	Serializes vb2 queue and video device operations.
+ * @slock:	Protect for pending_list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct v4l2_format vdev_fmt;
+	struct mtk_cam_dev_node_desc desc;
+	struct video_device vdev;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct list_head pending_list;
+	/* Used for vbq & vdev */
+	struct mutex lock;
+	/* protect for pending_list */
+	spinlock_t slock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @pdev:	Pointer to platform device
+ * @smem_pdev:	Pointer to shared memory platform device
+ * @pipeline:	Media pipeline information
+ * @media_dev:	Media device
+ * @subdev:	The V4L2 sub-device
+ * @v4l2_dev:	The V4L2 device driver
+ * @notifier:	The v4l2_device notifier data
+ * @subdev_pads: Pointer to the number of media pads of this sub-device
+ * @ctrl_handler: The control handler
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes
+ * @seninf:	Pointer to the seninf sub-device
+ * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
+ * @lock:       The mutex protecting video device open/release operations
+ * @streaming:	Indicate the overall streaming status is on or off
+ * @streamed_node_count: The number of V4L2 video device nodes are streaming on
+ * @req_list:	Lins to keep media requests before streaming on
+ * @req_lock:	Protect the req_list data
+ *
+ * Below is the graph topology for Camera IO connection.
+ * sensor 1 (main) --> sensor IF --> P1 sub-device
+ * sensor 2 (sub)  -->
+ *
+ */
+struct mtk_cam_dev {
+	struct platform_device *pdev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct v4l2_ctrl_handler ctrl_handler;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	/* protect video device open/release operations */
+	struct mutex lock;
+	unsigned int streaming:1;
+	atomic_t streamed_node_count;
+	struct list_head req_list;
+	/* protect for req_list */
+	spinlock_t req_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+int mtk_cam_dev_release(struct platform_device *pdev,
+			struct mtk_cam_dev *cam_dev);
+#endif /* __MTK_CAM_DEV_V4L2_H__ */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
 4 files changed, 1457 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 7558593e63f0..30df10983f6a 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -2,5 +2,6 @@
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
+mtk-cam-isp-objs += mtk_cam.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..9e59a6bfc6b7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT			BIT(0)
+#define CMOS_EN_BIT			BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST			BIT(0)
+#define HW_PASS1_DON_ST		BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define SW_PASS1_DON_ST		BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST			BIT(4)
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+#define PSO_DONE_ST			BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM		( \
+					VS_INT_ST |\
+					SOF_INT_ST |\
+					HW_PASS1_DON_ST |\
+					SW_PASS1_DON_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM		( \
+					SOF_INT_ST |\
+					SW_PASS1_DON_ST |\
+					HW_PASS1_DON_ST |\
+					VS_INT_ST |\
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					RRZO_ERR_ST |\
+					AFO_ERR_ST |\
+					IMGO_ERR_ST |\
+					AAO_ERR_ST |\
+					DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM		( \
+					AAO_DONE_ST |\
+					AFO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN			0x0004
+#define REG_CTL_DMA_EN			0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2			0x0018
+#define REG_CTL_RAW_INT_EN		0x0020
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define REG_CTL_RAW_INT2_STAT		0x0034
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_VF_CON			0x0234
+
+#define REG_IMGO_BASE_ADDR		0x1020
+#define REG_RRZO_BASE_ADDR		0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT		0x1360
+#define REG_RRZO_ERR_STAT		0x1364
+#define REG_AAO_ERR_STAT		0x1368
+#define REG_AFO_ERR_STAT		0x136c
+#define REG_LCSO_ERR_STAT		0x1370
+#define REG_UFEO_ERR_STAT		0x1374
+#define REG_PDO_ERR_STAT		0x1378
+#define REG_BPCI_ERR_STAT		0x137c
+#define REG_LSCI_ERR_STAT		0x1384
+#define REG_PDI_ERR_STAT		0x138c
+#define REG_LMVO_ERR_STAT		0x1390
+#define REG_FLKO_ERR_STAT		0x1394
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR		0x0198
+#define REG_HW_FRAME_NUM		0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX		0x14dc
+#define REG_META1_VB2_INDEX		0x151c
+
+/* FBC */
+#define REG_AAO_FBC_STATUS		0x013c
+#define REG_AFO_FBC_STATUS		0x0134
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..c5a3babed69d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1087 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* List of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+					 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+}
+
+static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
+				   unsigned int request_fd,
+				   unsigned int frame_seq_no,
+				   struct list_head *list_buf,
+				   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	u64    timestamp;
+
+	if (!cam_dev->streaming)
+		return;
+
+	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
+		__func__, request_fd, frame_seq_no, state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	timestamp = ktime_get_ns();
+	list_for_each_entry_safe(buf, b0, list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = timestamp;
+		buf->vbb.sequence = frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct vb2_buffer *vb;
+
+	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
+		return;
+
+	spin_lock(&node->slock);
+	b = list_first_entry(&node->pending_list,
+			     struct mtk_cam_dev_buffer,
+			     list);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (!vb->vb2_queue->uses_requests &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vb->timestamp = ktime_get_ns();
+			b->vbb.sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+			list_del(&b->list);
+			break;
+		}
+	}
+	spin_unlock(&node->slock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_warn(dev,
+				 "frame_seq_no:%d drop, queue_cnt:%d\n",
+				 framejob->frame_seq_no,
+				 atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		} else {
+			break;
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
+		    event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.dma_status_mask & AFO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int addr_offset;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	isp_dev->sof_count += 1;
+
+	if (cq_num <= frame_num) {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
+		return cq_num;
+	}
+	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
+
+	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(isp_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
+		cq_num, frame_num, addr_offset);
+
+	return cq_num;
+}
+
+static void irq_handle_notify_event(struct isp_device *isp_dev,
+				    unsigned int irq_status,
+				    unsigned int dma_status,
+				    bool sof_only)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned long flags;
+	int i;
+
+	if (irq_status & VS_INT_ST) {
+		/* Notify specific HW events to user space */
+		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
+					     isp_dev->current_frame);
+		dev_dbg(dev,
+			"frame sync is sent:%d:%d\n",
+			isp_dev->sof_count,
+			isp_dev->current_frame);
+		if (sof_only)
+			return;
+	}
+
+	if (irq_status & SW_PASS1_DON_ST) {
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask =
+		(irq_status & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask =
+		(dma_status & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irq_status & INT_ST_MASK_CAM),
+		(dma_status & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cam_idx, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned int aao_fbc, afo_fbc;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev.streaming)
+		return IRQ_HANDLED;
+
+	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
+	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
+		return IRQ_HANDLED;
+
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+
+	/* Sof, done order check */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
+		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue frame */
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irq_status & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
+	}
+
+	if (irq_status & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x\n",
+			irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irq_status & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cam_idx,
+			isp_dev->sof_count,
+			irq_status,
+			dma_status,
+			hw_frame_num,
+			cq_num,
+			aao_fbc,
+			afo_fbc);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+	mutex_init(&isp_ctx->lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+
+	return 0;
+}
+
+static int isp_uninit_context(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	if (isp_ctx->isp_deque_thread.thread) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	mutex_destroy(&isp_ctx->lock);
+
+	return 0;
+}
+
+static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enabled_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enabled_dma_ports = 0;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		if (cam_dev->vdev_nodes[i].enabled)
+			enabled_dma_ports |=
+				cam_dev->vdev_nodes[i].desc.dma_port;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
+
+	return enabled_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return RAW_PXL_ID_R;
+	default:
+		return RAW_PXL_ID_B;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return IMG_FMT_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return IMG_FMT_FG_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 10;
+	}
+}
+
+static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
+			   const struct v4l2_format *in_fmt,
+			   const struct v4l2_subdev_format *sd_format)
+{
+	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
+
+	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0x0;
+	out_fmt->crop.top = 0x0;
+
+	out_fmt->crop.width = sd_format->format.width;
+	out_fmt->crop.height = sd_format->format.height;
+
+	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
+		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
+		  "img out:%d:%d in:%d:%d",
+		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
+		  out_fmt->crop.width, out_fmt->crop.height);
+
+	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
+		out_fmt->pixel_byte,
+		out_fmt->img_fmt);
+	dev_dbg(dev,
+		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = isp_setup_scp_rproc(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_init_context(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_composer_init(dev);
+	if (ret)
+		goto composer_err;
+
+	pm_runtime_get_sync(dev);
+
+	/* ISP HW INIT */
+	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+	/* Use pure RAW as default HW path */
+	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
+
+	isp_composer_hw_init(dev);
+	/* Check enabled DMAs which is configured by media setup */
+	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+
+composer_err:
+	isp_uninit_context(dev);
+
+	return ret;
+}
+
+int mtk_isp_power_release(struct device *dev)
+{
+	isp_composer_hw_deinit(dev);
+	isp_uninit_context(dev);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_fmt;
+	unsigned int enabled_dma_ports;
+	struct v4l2_format *img_fmt;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
+
+	/* Sensor config */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
+
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_fmt.format.width, sd_fmt.format.height,
+		sd_fmt.format.code, sd_fmt.format.field,
+		sd_fmt.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* Fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = 0x1;
+	/* Support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_fmt.format.code);
+	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
+	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
+	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
+
+	config_param.cfg_main_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
+		config_param.cfg_main_param.bypass = 0;
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_img_fmt(dev, &config_param.cfg_main_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	config_param.cfg_resize_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
+		config_param.cfg_resize_param.bypass = 0;
+		config_img_fmt(dev, &config_param.cfg_resize_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
+
+	isp_composer_hw_config(dev, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer)
+{
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
+}
+
+void mtk_isp_req_flush_buffers(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_queue_job *job, *j0;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
+
+	if (!atomic_read(&p1_list->queue_cnt))
+		return;
+
+	spin_lock(&p1_list->lock);
+	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
+		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
+			list_del(&buf->list);
+			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(&buf->vbb.vb2_buf,
+						VB2_BUF_STATE_ERROR);
+		}
+		list_del(&job->list_entry);
+		atomic_dec(&p1_list->queue_cnt);
+		kfree(job);
+	}
+	spin_unlock(&p1_list->lock);
+}
+
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct media_request_object *obj, *obj_safe;
+	struct vb2_buffer *vb;
+	struct mtk_cam_dev_buffer *buf;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+		framejob->request_fd = buf->vbb.request_fd;
+		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
+		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
+		list_add_tail(&buf->list, &framejob->list_buf);
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		framejob->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s\n", __func__);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
+				      p1_dev->isp_ctx.clk_list);
+	if (ret)
+		goto clk_err;
+
+	return 0;
+
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* Disable view finder */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, the TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev.streaming) {
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq;
+	int ret;
+	unsigned int i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
+			i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* Support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* Reg & interrupts index is shifted with 1  */
+			irq = platform_get_irq(pdev, i - 1);
+			if (irq) {
+				ret = devm_request_irq(dev, irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						irq);
+					return ret;
+				}
+				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
+					irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_ctx.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_ctx.num_clks,
+			     sizeof(*p1_dev->isp_ctx.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_ctx.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
+		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_ctx.num_clks,
+				p1_dev->isp_ctx.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
+		goto err_init;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		goto err_init;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	pm_runtime_enable(dev);
+
+	return 0;
+
+err_init:
+	if (p1_dev->cam_dev.smem_dev)
+		device_unregister(p1_dev->cam_dev.smem_dev);
+
+	return ret;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-scp.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define CAM_A_MAX_WIDTH		3328
+#define CAM_A_MAX_HEIGHT		2496
+#define CAM_B_MAX_WIDTH		5376
+#define CAM_B_MAX_HEIGHT		4032
+
+#define CAM_MIN_WIDTH			80
+#define CAM_MIN_HEIGHT			60
+
+#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define R_IMGO				BIT(0)
+#define R_RRZO				BIT(1)
+#define R_AAO				BIT(3)
+#define R_AFO				BIT(4)
+#define R_LCSO				BIT(5)
+#define R_PDO				BIT(6)
+#define R_LMVO				BIT(7)
+#define R_FLKO				BIT(8)
+#define R_RSSO				BIT(9)
+#define R_PSO				BIT(10)
+
+#define CQ_BUFFER_COUNT		3
+#define IRQ_DATA_BUF_SIZE		4
+#define CQ_ADDRESS_OFFSET		0x640
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
+			"fbc1(0x%x), fbc2(0x%x)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+/* State for struct mtk_isp_p1_ctx: composer_state */
+enum  {
+	SCP_ON = 0,
+	SCP_OFF
+};
+
+enum {
+	IMG_FMT_UNKNOWN		= 0x0000,
+	IMG_FMT_RAW_START	= 0x2200,
+	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
+	IMG_FMT_BAYER10,
+	IMG_FMT_BAYER12,
+	IMG_FMT_BAYER14,
+	IMG_FMT_FG_BAYER8,
+	IMG_FMT_FG_BAYER10,
+	IMG_FMT_FG_BAYER12,
+	IMG_FMT_FG_BAYER14,
+};
+
+enum {
+	RAW_PXL_ID_B = 0,
+	RAW_PXL_ID_GB,
+	RAW_PXL_ID_GR,
+	RAW_PXL_ID_R
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+/*
+ * struct isp_device - the ISP device information
+ *
+ * @dev: Pointer to struct device
+ * @regs: Camera ISP base register address
+ * @spinlock_irq: Used to protect register read/write data
+ * @current_frame: Current frame sequence number, set when SOF
+ * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
+ * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
+ * @sof_count: The accumulated SOF counter
+ * @isp_hw_module: Identity camera A or B
+ *
+ */
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+/*
+ * struct mtk_isp_p1_ctx - the ISP device information
+ *
+ * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
+ * @composer_tx_thread: TX Thread for SCP data tranmission
+ * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
+ * @ipi_occupied: The total number of SCP TX data has beent sent
+ * @scp_state: The state of SCP control
+ * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
+ * @composed_frame_id: The ack. frame sequence by SCP
+ * @composer_deinit_thread: The de-initialized thread
+ * @p1_enqueue_list: Queue for ISP frame buffers
+ * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
+ * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
+ * @irq_data_start: Start index of irq_event_datas ring buffer
+ * @irq_data_end: End index of irq_event_datas ring buffer
+ * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
+ * @scp_mem_pa: DMA address for SCP device
+ * @scp_mem_iova: DMA address for ISP HW DMA devices
+ * @frame_seq_no: Sequence number for ISP frame buffer
+ * @isp_hw_module: Active camera HW module
+ * @num_clks: The number of driver's clock
+ * @clk_list: The list of clock data
+ * @lock: Lock to protect context operations
+ *
+ */
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	atomic_t ipi_occupied;
+	atomic_t scp_state;
+	atomic_t composing_frame;
+	atomic_t composed_frame_id;
+	struct isp_thread composer_deinit_thread;
+	struct isp_queue p1_enqueue_list;
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int num_clks;
+	struct clk_bulk_data *clk_list;
+	struct mutex lock; /* Protect context operations */
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct mtk_cam_dev cam_dev;
+	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_power_release(struct device *dev);
+int mtk_isp_config(struct device *dev);
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req);
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer);
+void mtk_isp_req_flush_buffers(struct device *dev);
+
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
 4 files changed, 1457 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 7558593e63f0..30df10983f6a 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -2,5 +2,6 @@
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
+mtk-cam-isp-objs += mtk_cam.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..9e59a6bfc6b7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT			BIT(0)
+#define CMOS_EN_BIT			BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST			BIT(0)
+#define HW_PASS1_DON_ST		BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define SW_PASS1_DON_ST		BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST			BIT(4)
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+#define PSO_DONE_ST			BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM		( \
+					VS_INT_ST |\
+					SOF_INT_ST |\
+					HW_PASS1_DON_ST |\
+					SW_PASS1_DON_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM		( \
+					SOF_INT_ST |\
+					SW_PASS1_DON_ST |\
+					HW_PASS1_DON_ST |\
+					VS_INT_ST |\
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					RRZO_ERR_ST |\
+					AFO_ERR_ST |\
+					IMGO_ERR_ST |\
+					AAO_ERR_ST |\
+					DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM		( \
+					AAO_DONE_ST |\
+					AFO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN			0x0004
+#define REG_CTL_DMA_EN			0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2			0x0018
+#define REG_CTL_RAW_INT_EN		0x0020
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define REG_CTL_RAW_INT2_STAT		0x0034
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_VF_CON			0x0234
+
+#define REG_IMGO_BASE_ADDR		0x1020
+#define REG_RRZO_BASE_ADDR		0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT		0x1360
+#define REG_RRZO_ERR_STAT		0x1364
+#define REG_AAO_ERR_STAT		0x1368
+#define REG_AFO_ERR_STAT		0x136c
+#define REG_LCSO_ERR_STAT		0x1370
+#define REG_UFEO_ERR_STAT		0x1374
+#define REG_PDO_ERR_STAT		0x1378
+#define REG_BPCI_ERR_STAT		0x137c
+#define REG_LSCI_ERR_STAT		0x1384
+#define REG_PDI_ERR_STAT		0x138c
+#define REG_LMVO_ERR_STAT		0x1390
+#define REG_FLKO_ERR_STAT		0x1394
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR		0x0198
+#define REG_HW_FRAME_NUM		0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX		0x14dc
+#define REG_META1_VB2_INDEX		0x151c
+
+/* FBC */
+#define REG_AAO_FBC_STATUS		0x013c
+#define REG_AFO_FBC_STATUS		0x0134
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..c5a3babed69d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1087 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* List of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+					 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+}
+
+static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
+				   unsigned int request_fd,
+				   unsigned int frame_seq_no,
+				   struct list_head *list_buf,
+				   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	u64    timestamp;
+
+	if (!cam_dev->streaming)
+		return;
+
+	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
+		__func__, request_fd, frame_seq_no, state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	timestamp = ktime_get_ns();
+	list_for_each_entry_safe(buf, b0, list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = timestamp;
+		buf->vbb.sequence = frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct vb2_buffer *vb;
+
+	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
+		return;
+
+	spin_lock(&node->slock);
+	b = list_first_entry(&node->pending_list,
+			     struct mtk_cam_dev_buffer,
+			     list);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (!vb->vb2_queue->uses_requests &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vb->timestamp = ktime_get_ns();
+			b->vbb.sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+			list_del(&b->list);
+			break;
+		}
+	}
+	spin_unlock(&node->slock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_warn(dev,
+				 "frame_seq_no:%d drop, queue_cnt:%d\n",
+				 framejob->frame_seq_no,
+				 atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		} else {
+			break;
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
+		    event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.dma_status_mask & AFO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int addr_offset;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	isp_dev->sof_count += 1;
+
+	if (cq_num <= frame_num) {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
+		return cq_num;
+	}
+	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
+
+	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(isp_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
+		cq_num, frame_num, addr_offset);
+
+	return cq_num;
+}
+
+static void irq_handle_notify_event(struct isp_device *isp_dev,
+				    unsigned int irq_status,
+				    unsigned int dma_status,
+				    bool sof_only)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned long flags;
+	int i;
+
+	if (irq_status & VS_INT_ST) {
+		/* Notify specific HW events to user space */
+		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
+					     isp_dev->current_frame);
+		dev_dbg(dev,
+			"frame sync is sent:%d:%d\n",
+			isp_dev->sof_count,
+			isp_dev->current_frame);
+		if (sof_only)
+			return;
+	}
+
+	if (irq_status & SW_PASS1_DON_ST) {
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask =
+		(irq_status & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask =
+		(dma_status & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irq_status & INT_ST_MASK_CAM),
+		(dma_status & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cam_idx, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned int aao_fbc, afo_fbc;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev.streaming)
+		return IRQ_HANDLED;
+
+	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
+	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
+		return IRQ_HANDLED;
+
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+
+	/* Sof, done order check */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
+		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue frame */
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irq_status & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
+	}
+
+	if (irq_status & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x\n",
+			irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irq_status & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cam_idx,
+			isp_dev->sof_count,
+			irq_status,
+			dma_status,
+			hw_frame_num,
+			cq_num,
+			aao_fbc,
+			afo_fbc);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+	mutex_init(&isp_ctx->lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+
+	return 0;
+}
+
+static int isp_uninit_context(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	if (isp_ctx->isp_deque_thread.thread) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	mutex_destroy(&isp_ctx->lock);
+
+	return 0;
+}
+
+static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enabled_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enabled_dma_ports = 0;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		if (cam_dev->vdev_nodes[i].enabled)
+			enabled_dma_ports |=
+				cam_dev->vdev_nodes[i].desc.dma_port;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
+
+	return enabled_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return RAW_PXL_ID_R;
+	default:
+		return RAW_PXL_ID_B;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return IMG_FMT_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return IMG_FMT_FG_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 10;
+	}
+}
+
+static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
+			   const struct v4l2_format *in_fmt,
+			   const struct v4l2_subdev_format *sd_format)
+{
+	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
+
+	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0x0;
+	out_fmt->crop.top = 0x0;
+
+	out_fmt->crop.width = sd_format->format.width;
+	out_fmt->crop.height = sd_format->format.height;
+
+	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
+		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
+		  "img out:%d:%d in:%d:%d",
+		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
+		  out_fmt->crop.width, out_fmt->crop.height);
+
+	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
+		out_fmt->pixel_byte,
+		out_fmt->img_fmt);
+	dev_dbg(dev,
+		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = isp_setup_scp_rproc(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_init_context(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_composer_init(dev);
+	if (ret)
+		goto composer_err;
+
+	pm_runtime_get_sync(dev);
+
+	/* ISP HW INIT */
+	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+	/* Use pure RAW as default HW path */
+	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
+
+	isp_composer_hw_init(dev);
+	/* Check enabled DMAs which is configured by media setup */
+	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+
+composer_err:
+	isp_uninit_context(dev);
+
+	return ret;
+}
+
+int mtk_isp_power_release(struct device *dev)
+{
+	isp_composer_hw_deinit(dev);
+	isp_uninit_context(dev);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_fmt;
+	unsigned int enabled_dma_ports;
+	struct v4l2_format *img_fmt;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
+
+	/* Sensor config */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
+
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_fmt.format.width, sd_fmt.format.height,
+		sd_fmt.format.code, sd_fmt.format.field,
+		sd_fmt.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* Fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = 0x1;
+	/* Support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_fmt.format.code);
+	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
+	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
+	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
+
+	config_param.cfg_main_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
+		config_param.cfg_main_param.bypass = 0;
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_img_fmt(dev, &config_param.cfg_main_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	config_param.cfg_resize_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
+		config_param.cfg_resize_param.bypass = 0;
+		config_img_fmt(dev, &config_param.cfg_resize_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
+
+	isp_composer_hw_config(dev, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer)
+{
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
+}
+
+void mtk_isp_req_flush_buffers(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_queue_job *job, *j0;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
+
+	if (!atomic_read(&p1_list->queue_cnt))
+		return;
+
+	spin_lock(&p1_list->lock);
+	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
+		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
+			list_del(&buf->list);
+			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(&buf->vbb.vb2_buf,
+						VB2_BUF_STATE_ERROR);
+		}
+		list_del(&job->list_entry);
+		atomic_dec(&p1_list->queue_cnt);
+		kfree(job);
+	}
+	spin_unlock(&p1_list->lock);
+}
+
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct media_request_object *obj, *obj_safe;
+	struct vb2_buffer *vb;
+	struct mtk_cam_dev_buffer *buf;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+		framejob->request_fd = buf->vbb.request_fd;
+		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
+		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
+		list_add_tail(&buf->list, &framejob->list_buf);
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		framejob->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s\n", __func__);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
+				      p1_dev->isp_ctx.clk_list);
+	if (ret)
+		goto clk_err;
+
+	return 0;
+
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* Disable view finder */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, the TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev.streaming) {
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq;
+	int ret;
+	unsigned int i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
+			i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* Support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* Reg & interrupts index is shifted with 1  */
+			irq = platform_get_irq(pdev, i - 1);
+			if (irq) {
+				ret = devm_request_irq(dev, irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						irq);
+					return ret;
+				}
+				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
+					irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_ctx.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_ctx.num_clks,
+			     sizeof(*p1_dev->isp_ctx.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_ctx.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
+		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_ctx.num_clks,
+				p1_dev->isp_ctx.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
+		goto err_init;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		goto err_init;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	pm_runtime_enable(dev);
+
+	return 0;
+
+err_init:
+	if (p1_dev->cam_dev.smem_dev)
+		device_unregister(p1_dev->cam_dev.smem_dev);
+
+	return ret;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-scp.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define CAM_A_MAX_WIDTH		3328
+#define CAM_A_MAX_HEIGHT		2496
+#define CAM_B_MAX_WIDTH		5376
+#define CAM_B_MAX_HEIGHT		4032
+
+#define CAM_MIN_WIDTH			80
+#define CAM_MIN_HEIGHT			60
+
+#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define R_IMGO				BIT(0)
+#define R_RRZO				BIT(1)
+#define R_AAO				BIT(3)
+#define R_AFO				BIT(4)
+#define R_LCSO				BIT(5)
+#define R_PDO				BIT(6)
+#define R_LMVO				BIT(7)
+#define R_FLKO				BIT(8)
+#define R_RSSO				BIT(9)
+#define R_PSO				BIT(10)
+
+#define CQ_BUFFER_COUNT		3
+#define IRQ_DATA_BUF_SIZE		4
+#define CQ_ADDRESS_OFFSET		0x640
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
+			"fbc1(0x%x), fbc2(0x%x)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+/* State for struct mtk_isp_p1_ctx: composer_state */
+enum  {
+	SCP_ON = 0,
+	SCP_OFF
+};
+
+enum {
+	IMG_FMT_UNKNOWN		= 0x0000,
+	IMG_FMT_RAW_START	= 0x2200,
+	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
+	IMG_FMT_BAYER10,
+	IMG_FMT_BAYER12,
+	IMG_FMT_BAYER14,
+	IMG_FMT_FG_BAYER8,
+	IMG_FMT_FG_BAYER10,
+	IMG_FMT_FG_BAYER12,
+	IMG_FMT_FG_BAYER14,
+};
+
+enum {
+	RAW_PXL_ID_B = 0,
+	RAW_PXL_ID_GB,
+	RAW_PXL_ID_GR,
+	RAW_PXL_ID_R
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+/*
+ * struct isp_device - the ISP device information
+ *
+ * @dev: Pointer to struct device
+ * @regs: Camera ISP base register address
+ * @spinlock_irq: Used to protect register read/write data
+ * @current_frame: Current frame sequence number, set when SOF
+ * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
+ * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
+ * @sof_count: The accumulated SOF counter
+ * @isp_hw_module: Identity camera A or B
+ *
+ */
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+/*
+ * struct mtk_isp_p1_ctx - the ISP device information
+ *
+ * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
+ * @composer_tx_thread: TX Thread for SCP data tranmission
+ * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
+ * @ipi_occupied: The total number of SCP TX data has beent sent
+ * @scp_state: The state of SCP control
+ * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
+ * @composed_frame_id: The ack. frame sequence by SCP
+ * @composer_deinit_thread: The de-initialized thread
+ * @p1_enqueue_list: Queue for ISP frame buffers
+ * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
+ * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
+ * @irq_data_start: Start index of irq_event_datas ring buffer
+ * @irq_data_end: End index of irq_event_datas ring buffer
+ * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
+ * @scp_mem_pa: DMA address for SCP device
+ * @scp_mem_iova: DMA address for ISP HW DMA devices
+ * @frame_seq_no: Sequence number for ISP frame buffer
+ * @isp_hw_module: Active camera HW module
+ * @num_clks: The number of driver's clock
+ * @clk_list: The list of clock data
+ * @lock: Lock to protect context operations
+ *
+ */
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	atomic_t ipi_occupied;
+	atomic_t scp_state;
+	atomic_t composing_frame;
+	atomic_t composed_frame_id;
+	struct isp_thread composer_deinit_thread;
+	struct isp_queue p1_enqueue_list;
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int num_clks;
+	struct clk_bulk_data *clk_list;
+	struct mutex lock; /* Protect context operations */
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct mtk_cam_dev cam_dev;
+	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_power_release(struct device *dev);
+int mtk_isp_config(struct device *dev);
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req);
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer);
+void mtk_isp_req_flush_buffers(struct device *dev);
+
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other functions.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
 4 files changed, 1457 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 7558593e63f0..30df10983f6a 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -2,5 +2,6 @@
 
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
+mtk-cam-isp-objs += mtk_cam.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..9e59a6bfc6b7
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _CAM_REGS_H
+#define _CAM_REGS_H
+
+/* TG Bit Mask */
+#define VFDATA_EN_BIT			BIT(0)
+#define CMOS_EN_BIT			BIT(0)
+
+/* normal signal bit */
+#define VS_INT_ST			BIT(0)
+#define HW_PASS1_DON_ST		BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define SW_PASS1_DON_ST		BIT(30)
+
+/* err status bit */
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+
+/* CAM DMA done status */
+#define FLKO_DONE_ST			BIT(4)
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+#define PSO_DONE_ST			BIT(14)
+
+/* IRQ signal mask */
+#define INT_ST_MASK_CAM		( \
+					VS_INT_ST |\
+					SOF_INT_ST |\
+					HW_PASS1_DON_ST |\
+					SW_PASS1_DON_ST)
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+/* IRQ Signal Log Mask */
+#define INT_ST_LOG_MASK_CAM		( \
+					SOF_INT_ST |\
+					SW_PASS1_DON_ST |\
+					HW_PASS1_DON_ST |\
+					VS_INT_ST |\
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					RRZO_ERR_ST |\
+					AFO_ERR_ST |\
+					IMGO_ERR_ST |\
+					AAO_ERR_ST |\
+					DMA_ERR_ST)
+
+/* DMA Event Notification Mask */
+#define DMA_ST_MASK_CAM		( \
+					AAO_DONE_ST |\
+					AFO_DONE_ST)
+
+/* Status check */
+#define REG_CTL_EN			0x0004
+#define REG_CTL_DMA_EN			0x0008
+#define REG_CTL_FMT_SEL		0x0010
+#define REG_CTL_EN2			0x0018
+#define REG_CTL_RAW_INT_EN		0x0020
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define REG_CTL_RAW_INT2_STAT		0x0034
+
+#define REG_TG_SEN_MODE		0x0230
+#define REG_TG_VF_CON			0x0234
+
+#define REG_IMGO_BASE_ADDR		0x1020
+#define REG_RRZO_BASE_ADDR		0x1050
+
+/* Error status log */
+#define REG_IMGO_ERR_STAT		0x1360
+#define REG_RRZO_ERR_STAT		0x1364
+#define REG_AAO_ERR_STAT		0x1368
+#define REG_AFO_ERR_STAT		0x136c
+#define REG_LCSO_ERR_STAT		0x1370
+#define REG_UFEO_ERR_STAT		0x1374
+#define REG_PDO_ERR_STAT		0x1378
+#define REG_BPCI_ERR_STAT		0x137c
+#define REG_LSCI_ERR_STAT		0x1384
+#define REG_PDI_ERR_STAT		0x138c
+#define REG_LMVO_ERR_STAT		0x1390
+#define REG_FLKO_ERR_STAT		0x1394
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* ISP command */
+#define REG_CQ_THR0_BASEADDR		0x0198
+#define REG_HW_FRAME_NUM		0x13b8
+
+/* META */
+#define REG_META0_VB2_INDEX		0x14dc
+#define REG_META1_VB2_INDEX		0x151c
+
+/* FBC */
+#define REG_AAO_FBC_STATUS		0x013c
+#define REG_AFO_FBC_STATUS		0x0134
+
+#endif	/* _CAM_REGS_H */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..c5a3babed69d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,1087 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/cdev.h>
+#include <linux/compat.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched/clock.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-regs.h"
+#include "mtk_cam-smem.h"
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+/* List of clocks required by isp cam */
+static const char * const mtk_isp_clks[] = {
+	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+};
+
+static void isp_dump_dma_status(struct isp_device *isp_dev)
+{
+	dev_err(isp_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
+		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
+		readl(isp_dev->regs + REG_AAO_ERR_STAT),
+		readl(isp_dev->regs + REG_AFO_ERR_STAT),
+		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(isp_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
+		readl(isp_dev->regs + REG_PSO_ERR_STAT),
+		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
+		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
+		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
+}
+
+static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+					 __u32 frame_seq_no)
+{
+	struct v4l2_event event;
+
+	memset(&event, 0, sizeof(event));
+	event.type = V4L2_EVENT_FRAME_SYNC;
+	event.u.frame_sync.frame_sequence = frame_seq_no;
+	v4l2_event_queue(cam_dev->subdev.devnode, &event);
+}
+
+static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
+				   unsigned int request_fd,
+				   unsigned int frame_seq_no,
+				   struct list_head *list_buf,
+				   enum vb2_buffer_state state)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	u64    timestamp;
+
+	if (!cam_dev->streaming)
+		return;
+
+	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
+		__func__, request_fd, frame_seq_no, state);
+
+	/*
+	 * Set the buffer's VB2 status so that the user can dequeue
+	 * the buffer.
+	 */
+	timestamp = ktime_get_ns();
+	list_for_each_entry_safe(buf, b0, list_buf, list) {
+		list_del(&buf->list);
+		buf->vbb.vb2_buf.timestamp = timestamp;
+		buf->vbb.sequence = frame_seq_no;
+		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+			vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+static void isp_deque_frame(struct isp_p1_device *p1_dev,
+			    unsigned int node_id, int vb2_index,
+			    int frame_seq_no)
+{
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
+	struct mtk_cam_dev_buffer *b, *b0;
+	struct vb2_buffer *vb;
+
+	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
+		return;
+
+	spin_lock(&node->slock);
+	b = list_first_entry(&node->pending_list,
+			     struct mtk_cam_dev_buffer,
+			     list);
+	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
+		vb = &b->vbb.vb2_buf;
+		if (!vb->vb2_queue->uses_requests &&
+		    vb->index == vb2_index &&
+		    vb->state == VB2_BUF_STATE_ACTIVE) {
+			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
+			vb->timestamp = ktime_get_ns();
+			b->vbb.sequence = frame_seq_no;
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+			list_del(&b->list);
+			break;
+		}
+	}
+	spin_unlock(&node->slock);
+}
+
+static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
+				    int frame_seq_no)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_job *framejob, *tmp;
+	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
+
+	/* Match dequeue work and enqueue frame */
+	spin_lock(&p1_enqueue_list->lock);
+	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
+				 list_entry) {
+		dev_dbg(dev,
+			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
+			__func__,
+			framejob->frame_seq_no, frame_seq_no);
+		/* Match by the en-queued request number */
+		if (framejob->frame_seq_no == frame_seq_no) {
+			/* Pass to user space */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_DONE);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_dbg(dev,
+				"frame_seq_no:%d is done, queue_cnt:%d\n",
+				framejob->frame_seq_no,
+				atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only when frame ready */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+			break;
+		} else if (framejob->frame_seq_no < frame_seq_no) {
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_finish(isp_ctx,
+					       framejob->request_fd,
+					       framejob->frame_seq_no,
+					       &framejob->list_buf,
+					       VB2_BUF_STATE_ERROR);
+			atomic_dec(&p1_enqueue_list->queue_cnt);
+			dev_warn(dev,
+				 "frame_seq_no:%d drop, queue_cnt:%d\n",
+				 framejob->frame_seq_no,
+				 atomic_read(&p1_enqueue_list->queue_cnt));
+
+			/* Remove only drop frame */
+			list_del(&framejob->list_entry);
+			kfree(framejob);
+		} else {
+			break;
+		}
+	}
+	spin_unlock(&p1_enqueue_list->lock);
+}
+
+static int isp_deque_work(void *data)
+{
+	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_cam_dev_stat_event_data event_data;
+	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
+	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
+	unsigned long flags;
+	int ret, i;
+
+	while (1) {
+		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
+					       (atomic_read(irq_data_end) !=
+					       atomic_read(irq_data_start)) ||
+					       kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+		i = atomic_read(&isp_ctx->irq_data_start);
+		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
+		       sizeof(event_data));
+		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
+		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
+		    event_data.dma_status_mask & AAO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.dma_status_mask & AFO_DONE_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+		}
+		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_0,
+					event_data.meta0_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_frame(p1_dev,
+					MTK_CAM_P1_META_OUT_1,
+					event_data.meta1_vb2_index,
+					event_data.frame_seq_no);
+			isp_deque_request_frame(p1_dev,
+						event_data.frame_seq_no);
+		}
+	}
+
+	return 0;
+}
+
+static int irq_handle_sof(struct isp_device *isp_dev,
+			  dma_addr_t base_addr,
+			  unsigned int frame_num)
+{
+	unsigned int addr_offset;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
+
+	isp_dev->sof_count += 1;
+
+	if (cq_num <= frame_num) {
+		dev_dbg(isp_dev->dev,
+			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
+			cq_num, frame_num);
+		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
+		return cq_num;
+	}
+	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
+
+	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(isp_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
+		cq_num, frame_num, addr_offset);
+
+	return cq_num;
+}
+
+static void irq_handle_notify_event(struct isp_device *isp_dev,
+				    unsigned int irq_status,
+				    unsigned int dma_status,
+				    bool sof_only)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned long flags;
+	int i;
+
+	if (irq_status & VS_INT_ST) {
+		/* Notify specific HW events to user space */
+		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
+					     isp_dev->current_frame);
+		dev_dbg(dev,
+			"frame sync is sent:%d:%d\n",
+			isp_dev->sof_count,
+			isp_dev->current_frame);
+		if (sof_only)
+			return;
+	}
+
+	if (irq_status & SW_PASS1_DON_ST) {
+		/* Notify TX thread to send if TX frame is blocked */
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+	}
+
+	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
+	i = atomic_read(&isp_ctx->irq_data_end);
+	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
+	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
+	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
+	isp_ctx->irq_event_datas[i].irq_status_mask =
+		(irq_status & INT_ST_MASK_CAM);
+	isp_ctx->irq_event_datas[i].dma_status_mask =
+		(dma_status & DMA_ST_MASK_CAM);
+	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
+	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
+
+	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+
+	dev_dbg(dev,
+		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
+		__func__,
+		(irq_status & INT_ST_MASK_CAM),
+		(dma_status & DMA_ST_MASK_CAM),
+		isp_dev->current_frame,
+		isp_dev->meta0_vb2_index,
+		isp_dev->meta1_vb2_index);
+}
+
+irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct isp_device *isp_dev = (struct isp_device *)data;
+	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = isp_dev->dev;
+	unsigned int cam_idx, cq_num, hw_frame_num;
+	unsigned int meta0_vb2_index, meta1_vb2_index;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned int aao_fbc, afo_fbc;
+	unsigned long flags;
+
+	/* Check the streaming is off or not */
+	if (!p1_dev->cam_dev.streaming)
+		return IRQ_HANDLED;
+
+	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
+	cq_num = 0;
+
+	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
+	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
+	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
+	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
+	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
+	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
+	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
+	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
+	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
+
+	/* Ignore unnecessary IRQ */
+	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
+		return IRQ_HANDLED;
+
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+
+	/* Sof, done order check */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
+		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
+
+		/* Notify IRQ event and enqueue frame */
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
+		isp_dev->current_frame = hw_frame_num;
+		isp_dev->meta0_vb2_index = meta0_vb2_index;
+		isp_dev->meta1_vb2_index = meta1_vb2_index;
+	} else {
+		if (irq_status & SOF_INT_ST) {
+			isp_dev->current_frame = hw_frame_num;
+			isp_dev->meta0_vb2_index = meta0_vb2_index;
+			isp_dev->meta1_vb2_index = meta1_vb2_index;
+		}
+		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
+	}
+
+	if (irq_status & SOF_INT_ST)
+		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
+					hw_frame_num);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev,
+			"raw_int_err:0x%x/0x%x\n",
+			irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_dump_dma_status(isp_dev);
+	}
+
+	if (irq_status & INT_ST_LOG_MASK_CAM)
+		dev_dbg(dev, IRQ_STAT_STR,
+			'A' + cam_idx,
+			isp_dev->sof_count,
+			irq_status,
+			dma_status,
+			hw_frame_num,
+			cq_num,
+			aao_fbc,
+			afo_fbc);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
+{
+	phandle rproc_phandle;
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "Failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "fail to get rproc_handle\n");
+		return -EINVAL;
+	}
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		/*
+		 * Return 0 if downloading firmware successfully,
+		 * otherwise it is failed
+		 */
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int isp_init_context(struct isp_p1_device *p1_dev)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct device *dev = &p1_dev->pdev->dev;
+	unsigned int i;
+
+	dev_dbg(dev, "init irq work thread\n");
+	if (!isp_ctx->isp_deque_thread.thread) {
+		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread =
+			kthread_run(isp_deque_work, (void *)p1_dev,
+				    "isp_deque_work");
+		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
+			dev_err(dev, "unable to alloc kthread\n");
+			isp_ctx->isp_deque_thread.thread = NULL;
+			return -ENOMEM;
+		}
+	}
+	spin_lock_init(&isp_ctx->irq_dequeue_lock);
+	mutex_init(&isp_ctx->lock);
+
+	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
+	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
+
+	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
+		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	spin_lock_init(&isp_ctx->composer_txlist.lock);
+
+	atomic_set(&isp_ctx->irq_data_end, 0);
+	atomic_set(&isp_ctx->irq_data_start, 0);
+
+	return 0;
+}
+
+static int isp_uninit_context(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct mtk_isp_queue_job *framejob, *tmp_framejob;
+
+	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
+	list_for_each_entry_safe(framejob, tmp_framejob,
+				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
+		list_del(&framejob->list_entry);
+		kfree(framejob);
+	}
+	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
+
+	if (isp_ctx->isp_deque_thread.thread) {
+		kthread_stop(isp_ctx->isp_deque_thread.thread);
+		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
+		isp_ctx->isp_deque_thread.thread = NULL;
+	}
+
+	mutex_destroy(&isp_ctx->lock);
+
+	return 0;
+}
+
+static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
+{
+	unsigned int enabled_dma_ports, i;
+
+	/* Get the enabled meta DMA ports */
+	enabled_dma_ports = 0;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		if (cam_dev->vdev_nodes[i].enabled)
+			enabled_dma_ports |=
+				cam_dev->vdev_nodes[i].desc.dma_port;
+
+	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
+
+	return enabled_dma_ports;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return RAW_PXL_ID_R;
+	default:
+		return RAW_PXL_ID_B;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return IMG_FMT_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+		return IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_F8:
+		return IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_B10:
+		return IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_F10:
+		return IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_B12:
+		return IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_F12:
+		return IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_B14:
+		return IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_F14:
+		return IMG_FMT_FG_BAYER14;
+	default:
+		return IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_pixel_byte(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_B8:
+	case V4L2_PIX_FMT_MTISP_F8:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_B10:
+	case V4L2_PIX_FMT_MTISP_F10:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_B12:
+	case V4L2_PIX_FMT_MTISP_F12:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_B14:
+	case V4L2_PIX_FMT_MTISP_F14:
+		return 14;
+	default:
+		return 10;
+	}
+}
+
+static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
+			   const struct v4l2_format *in_fmt,
+			   const struct v4l2_subdev_format *sd_format)
+{
+	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
+
+	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0x0;
+	out_fmt->crop.top = 0x0;
+
+	out_fmt->crop.width = sd_format->format.width;
+	out_fmt->crop.height = sd_format->format.height;
+
+	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
+		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
+		  "img out:%d:%d in:%d:%d",
+		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
+		  out_fmt->crop.width, out_fmt->crop.height);
+
+	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
+		out_fmt->pixel_byte,
+		out_fmt->img_fmt);
+	dev_dbg(dev,
+		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+}
+
+/* ISP P1 interface functions */
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
+{
+	struct device *dev = &cam_dev->pdev->dev;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = isp_setup_scp_rproc(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_init_context(p1_dev);
+	if (ret)
+		return ret;
+
+	ret = isp_composer_init(dev);
+	if (ret)
+		goto composer_err;
+
+	pm_runtime_get_sync(dev);
+
+	/* ISP HW INIT */
+	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
+	/* Use pure RAW as default HW path */
+	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
+	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
+
+	isp_composer_hw_init(dev);
+	/* Check enabled DMAs which is configured by media setup */
+	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+
+composer_err:
+	isp_uninit_context(dev);
+
+	return ret;
+}
+
+int mtk_isp_power_release(struct device *dev)
+{
+	isp_composer_hw_deinit(dev);
+	isp_uninit_context(dev);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_config(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_config_param config_param;
+	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
+	struct v4l2_subdev_format sd_fmt;
+	unsigned int enabled_dma_ports;
+	struct v4l2_format *img_fmt;
+	int ret;
+
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
+	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
+
+	isp_ctx->frame_seq_no = 1;
+	atomic_set(&isp_ctx->composed_frame_id, 0);
+
+	/* Get the enabled DMA ports */
+	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
+	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
+
+	/* Sensor config */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
+
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
+		return -EPERM;
+	}
+
+	dev_dbg(dev,
+		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
+		ret, sd_fmt.format.width, sd_fmt.format.height,
+		sd_fmt.format.code, sd_fmt.format.field,
+		sd_fmt.format.colorspace);
+
+	config_param.cfg_in_param.continuous = 0x1;
+	config_param.cfg_in_param.subsample = 0x0;
+	/* Fix to one pixel mode in default */
+	config_param.cfg_in_param.pixel_mode = 0x1;
+	/* Support normal pattern in default */
+	config_param.cfg_in_param.data_pattern = 0x0;
+
+	config_param.cfg_in_param.crop.left = 0x0;
+	config_param.cfg_in_param.crop.top = 0x0;
+
+	config_param.cfg_in_param.raw_pixel_id =
+		get_sensor_pixel_id(sd_fmt.format.code);
+	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
+	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
+	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
+
+	config_param.cfg_main_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
+		config_param.cfg_main_param.bypass = 0;
+		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
+		config_param.cfg_main_param.pure_raw_pack = 1;
+		config_img_fmt(dev, &config_param.cfg_main_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	config_param.cfg_resize_param.bypass = 1;
+	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
+	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
+		config_param.cfg_resize_param.bypass = 0;
+		config_img_fmt(dev, &config_param.cfg_resize_param.output,
+			       img_fmt, &sd_fmt);
+	}
+
+	/* Configure meta DMAs info. */
+	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
+
+	isp_composer_hw_config(dev, &config_param);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer)
+{
+	struct mtk_isp_scp_p1_cmd frameparams;
+
+	memset(&frameparams, 0, sizeof(frameparams));
+	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
+	frameparams.meta_frame.enabled_dma = dma_port;
+	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
+	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
+	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
+}
+
+void mtk_isp_req_flush_buffers(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_queue_job *job, *j0;
+	struct mtk_cam_dev_buffer *buf, *b0;
+	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
+
+	if (!atomic_read(&p1_list->queue_cnt))
+		return;
+
+	spin_lock(&p1_list->lock);
+	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
+		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
+			list_del(&buf->list);
+			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+				vb2_buffer_done(&buf->vbb.vb2_buf,
+						VB2_BUF_STATE_ERROR);
+		}
+		list_del(&job->list_entry);
+		atomic_dec(&p1_list->queue_cnt);
+		kfree(job);
+	}
+	spin_unlock(&p1_list->lock);
+}
+
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	struct p1_frame_param frameparams;
+	struct mtk_isp_queue_job *framejob;
+	struct media_request_object *obj, *obj_safe;
+	struct vb2_buffer *vb;
+	struct mtk_cam_dev_buffer *buf;
+
+	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
+	memset(framejob, 0, sizeof(*framejob));
+	memset(&frameparams, 0, sizeof(frameparams));
+	INIT_LIST_HEAD(&framejob->list_buf);
+
+	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
+	frameparams.sof_idx =
+		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
+	framejob->frame_seq_no = frameparams.frame_seq_no;
+
+	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+		framejob->request_fd = buf->vbb.request_fd;
+		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
+		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
+		list_add_tail(&buf->list, &framejob->list_buf);
+	}
+
+	spin_lock(&isp_ctx->p1_enqueue_list.lock);
+	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
+	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
+	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
+
+	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
+	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
+		framejob->request_fd,
+		frameparams.frame_seq_no,
+		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
+}
+
+static int enable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	int ret;
+
+	dev_info(dev, "- %s\n", __func__);
+
+	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
+				      p1_dev->isp_ctx.clk_list);
+	if (ret)
+		goto clk_err;
+
+	return 0;
+
+clk_err:
+	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+	return ret;
+}
+
+static void disable_sys_clock(struct isp_p1_device *p1_dev)
+{
+	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
+				   p1_dev->isp_ctx.clk_list);
+}
+
+static int mtk_isp_suspend(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	isp_dev = &p1_dev->isp_devs[module];
+	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+	if (reg_val & VFDATA_EN_BIT) {
+		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
+		/* Disable view finder */
+		writel((reg_val & (~VFDATA_EN_BIT)),
+		       isp_dev->regs + REG_TG_VF_CON);
+		/*
+		 * After VF enable, the TG frame count will be reset to 0;
+		 */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val & (~CMOS_EN_BIT)),
+		       isp_dev->regs +  + REG_TG_SEN_MODE);
+	}
+
+	disable_sys_clock(p1_dev);
+
+	return 0;
+}
+
+static int mtk_isp_resume(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	int module = p1_dev->isp_ctx.isp_hw_module;
+	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
+	unsigned int reg_val;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	enable_sys_clock(p1_dev);
+
+	/* V4L2 stream-on phase & restore HW stream-on status */
+	if (p1_dev->cam_dev.streaming) {
+		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
+		/* Enable CMOS */
+		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
+		writel((reg_val | CMOS_EN_BIT),
+		       isp_dev->regs + REG_TG_SEN_MODE);
+		/* Enable VF */
+		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
+		writel((reg_val | VFDATA_EN_BIT),
+		       isp_dev->regs + REG_TG_VF_CON);
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	struct isp_p1_device *p1_dev;
+	struct mtk_isp_p1_ctx *isp_ctx;
+	struct isp_device *isp_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq;
+	int ret;
+	unsigned int i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	dev_set_drvdata(dev, p1_dev);
+	isp_ctx = &p1_dev->isp_ctx;
+	p1_dev->pdev = pdev;
+
+	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
+		isp_dev = &p1_dev->isp_devs[i];
+		isp_dev->isp_hw_module = i;
+		isp_dev->dev = dev;
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		isp_dev->regs = devm_ioremap_resource(dev, res);
+
+		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
+			i, (unsigned long)isp_dev->regs);
+
+		if (!isp_dev->regs)
+			return PTR_ERR(isp_dev->regs);
+
+		/* Support IRQ from ISP_CAM_A_IDX */
+		if (i >= ISP_CAM_A_IDX) {
+			/* Reg & interrupts index is shifted with 1  */
+			irq = platform_get_irq(pdev, i - 1);
+			if (irq) {
+				ret = devm_request_irq(dev, irq,
+						       isp_irq_cam,
+						       IRQF_SHARED,
+						       dev_driver_string(dev),
+						       (void *)isp_dev);
+				if (ret) {
+					dev_err(dev,
+						"req_irq fail, dev:%s irq=%d\n",
+						dev->of_node->name,
+						irq);
+					return ret;
+				}
+				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
+					irq, dev_driver_string(dev));
+			}
+		}
+		spin_lock_init(&isp_dev->spinlock_irq);
+	}
+
+	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
+	p1_dev->isp_ctx.clk_list =
+		devm_kcalloc(dev,
+			     p1_dev->isp_ctx.num_clks,
+			     sizeof(*p1_dev->isp_ctx.clk_list),
+			     GFP_KERNEL);
+	if (!p1_dev->isp_ctx.clk_list)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
+		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
+
+	ret = devm_clk_bulk_get(dev,
+				p1_dev->isp_ctx.num_clks,
+				p1_dev->isp_ctx.clk_list);
+	if (ret) {
+		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize reserved DMA memory */
+	ret = mtk_cam_reserved_memory_init(p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
+		goto err_init;
+	}
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		goto err_init;
+
+	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
+	pm_runtime_enable(dev);
+
+	return 0;
+
+err_init:
+	if (p1_dev->cam_dev.smem_dev)
+		device_unregister(p1_dev->cam_dev.smem_dev);
+
+	return ret;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	pm_runtime_disable(dev);
+	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
+};
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Camera ISP driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __CAMERA_ISP_H
+#define __CAMERA_ISP_H
+
+#include <linux/cdev.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/ioctl.h>
+#include <linux/irqreturn.h>
+#include <linux/miscdevice.h>
+#include <linux/pm_qos.h>
+#include <linux/scatterlist.h>
+
+#include "mtk_cam-scp.h"
+#include "mtk_cam-v4l2-util.h"
+
+#define CAM_A_MAX_WIDTH		3328
+#define CAM_A_MAX_HEIGHT		2496
+#define CAM_B_MAX_WIDTH		5376
+#define CAM_B_MAX_HEIGHT		4032
+
+#define CAM_MIN_WIDTH			80
+#define CAM_MIN_HEIGHT			60
+
+#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
+#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
+#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
+#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
+#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
+
+#define R_IMGO				BIT(0)
+#define R_RRZO				BIT(1)
+#define R_AAO				BIT(3)
+#define R_AFO				BIT(4)
+#define R_LCSO				BIT(5)
+#define R_PDO				BIT(6)
+#define R_LMVO				BIT(7)
+#define R_FLKO				BIT(8)
+#define R_RSSO				BIT(9)
+#define R_PSO				BIT(10)
+
+#define CQ_BUFFER_COUNT		3
+#define IRQ_DATA_BUF_SIZE		4
+#define CQ_ADDRESS_OFFSET		0x640
+
+#define ISP_COMPOSING_MAX_NUM		4
+#define ISP_FRAME_COMPOSING_MAX_NUM	3
+
+#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
+			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
+			"fbc1(0x%x), fbc2(0x%x)\n"
+
+/*
+ * In order with the sequence of device nodes defined in dtsi rule,
+ * one hardware module should be mapping to one node.
+ */
+enum isp_dev_node_enum {
+	ISP_CAMSYS_CONFIG_IDX = 0,
+	ISP_CAM_UNI_IDX,
+	ISP_CAM_A_IDX,
+	ISP_CAM_B_IDX,
+	ISP_DEV_NODE_NUM
+};
+
+/* Image RAW path for ISP P1 module. */
+enum isp_raw_path_enum {
+	ISP_PROCESS_RAW_PATH = 0,
+	ISP_PURE_RAW_PATH
+};
+
+/* State for struct mtk_isp_p1_ctx: composer_state */
+enum  {
+	SCP_ON = 0,
+	SCP_OFF
+};
+
+enum {
+	IMG_FMT_UNKNOWN		= 0x0000,
+	IMG_FMT_RAW_START	= 0x2200,
+	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
+	IMG_FMT_BAYER10,
+	IMG_FMT_BAYER12,
+	IMG_FMT_BAYER14,
+	IMG_FMT_FG_BAYER8,
+	IMG_FMT_FG_BAYER10,
+	IMG_FMT_FG_BAYER12,
+	IMG_FMT_FG_BAYER14,
+};
+
+enum {
+	RAW_PXL_ID_B = 0,
+	RAW_PXL_ID_GB,
+	RAW_PXL_ID_GR,
+	RAW_PXL_ID_R
+};
+
+struct isp_queue {
+	struct list_head queue;
+	atomic_t queue_cnt;
+	spinlock_t lock; /* queue attributes protection */
+};
+
+struct isp_thread {
+	struct task_struct *thread;
+	wait_queue_head_t wq;
+};
+
+struct mtk_isp_queue_work {
+	union {
+		struct mtk_isp_scp_p1_cmd cmd;
+		struct p1_frame_param frameparams;
+	};
+	struct list_head list_entry;
+	enum mtk_isp_scp_type type;
+};
+
+struct mtk_cam_dev_stat_event_data {
+	__u32 frame_seq_no;
+	__u32 meta0_vb2_index;
+	__u32 meta1_vb2_index;
+	__u32 irq_status_mask;
+	__u32 dma_status_mask;
+};
+
+struct mtk_isp_queue_job {
+	struct list_head list_entry;
+	struct list_head list_buf;
+	unsigned int request_fd;
+	unsigned int frame_seq_no;
+};
+
+/*
+ * struct isp_device - the ISP device information
+ *
+ * @dev: Pointer to struct device
+ * @regs: Camera ISP base register address
+ * @spinlock_irq: Used to protect register read/write data
+ * @current_frame: Current frame sequence number, set when SOF
+ * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
+ * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
+ * @sof_count: The accumulated SOF counter
+ * @isp_hw_module: Identity camera A or B
+ *
+ */
+struct isp_device {
+	struct device *dev;
+	void __iomem *regs;
+	spinlock_t spinlock_irq; /* ISP reg setting integrity */
+	unsigned int current_frame;
+	unsigned int meta0_vb2_index;
+	unsigned int meta1_vb2_index;
+	u8 sof_count;
+	u8 isp_hw_module;
+};
+
+/*
+ * struct mtk_isp_p1_ctx - the ISP device information
+ *
+ * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
+ * @composer_tx_thread: TX Thread for SCP data tranmission
+ * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
+ * @ipi_occupied: The total number of SCP TX data has beent sent
+ * @scp_state: The state of SCP control
+ * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
+ * @composed_frame_id: The ack. frame sequence by SCP
+ * @composer_deinit_thread: The de-initialized thread
+ * @p1_enqueue_list: Queue for ISP frame buffers
+ * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
+ * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
+ * @irq_data_start: Start index of irq_event_datas ring buffer
+ * @irq_data_end: End index of irq_event_datas ring buffer
+ * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
+ * @scp_mem_pa: DMA address for SCP device
+ * @scp_mem_iova: DMA address for ISP HW DMA devices
+ * @frame_seq_no: Sequence number for ISP frame buffer
+ * @isp_hw_module: Active camera HW module
+ * @num_clks: The number of driver's clock
+ * @clk_list: The list of clock data
+ * @lock: Lock to protect context operations
+ *
+ */
+struct mtk_isp_p1_ctx {
+	struct isp_queue composer_txlist;
+	struct isp_thread composer_tx_thread;
+	atomic_t cmd_queued;
+	atomic_t ipi_occupied;
+	atomic_t scp_state;
+	atomic_t composing_frame;
+	atomic_t composed_frame_id;
+	struct isp_thread composer_deinit_thread;
+	struct isp_queue p1_enqueue_list;
+	struct isp_thread isp_deque_thread;
+	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
+	atomic_t irq_data_start;
+	atomic_t irq_data_end;
+	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
+	dma_addr_t scp_mem_pa;
+	dma_addr_t scp_mem_iova;
+	int frame_seq_no;
+	unsigned int isp_hw_module;
+	unsigned int isp_raw_path;
+	unsigned int num_clks;
+	struct clk_bulk_data *clk_list;
+	struct mutex lock; /* Protect context operations */
+};
+
+struct isp_p1_device {
+	struct platform_device *pdev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_isp_p1_ctx isp_ctx;
+	struct mtk_cam_dev cam_dev;
+	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
+};
+
+static inline struct isp_p1_device *
+p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
+{
+	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
+}
+
+static inline struct isp_p1_device *get_p1_device(struct device *dev)
+{
+	return ((struct isp_p1_device *)dev_get_drvdata(dev));
+}
+
+int mtk_isp_power_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_power_release(struct device *dev);
+int mtk_isp_config(struct device *dev);
+void mtk_isp_req_enqueue(struct device *dev, struct media_request *req);
+void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
+		     struct mtk_cam_dev_buffer *buffer);
+void mtk_isp_req_flush_buffers(struct device *dev);
+
+#endif /*__CAMERA_ISP_H*/
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
 3 files changed, 579 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 30df10983f6a..95f0b1c8fa1c 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -3,5 +3,6 @@
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-scp.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..04519d0b942f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,371 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+
+	spin_lock(&isp_ctx->composer_txlist.lock);
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	spin_unlock(&isp_ctx->composer_txlist.lock);
+
+	mutex_lock(&isp_ctx->lock);
+	if (isp_ctx->composer_tx_thread.thread) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	if (isp_ctx->composer_deinit_thread.thread) {
+		wake_up(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread = NULL;
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM) ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_deinit_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
+				 kthread_should_stop());
+
+	dev_dbg(dev, "%s run deinit", __func__);
+	isp_composer_deinit(isp_ctx);
+
+	return 0;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK)
+		return;
+
+	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
+		dev_dbg(dev, "ack frame_num:%d",
+			ipi_msg->ack_info.frame_seq_no);
+		atomic_set(&isp_ctx->composed_frame_id,
+			   ipi_msg->ack_info.frame_seq_no);
+	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
+		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
+		atomic_set(&isp_ctx->scp_state, SCP_OFF);
+		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
+	}
+
+	atomic_dec_return(&isp_ctx->ipi_occupied);
+	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+}
+
+int isp_composer_init(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_ON);
+
+	mutex_lock(&isp_ctx->lock);
+	if (!isp_ctx->composer_tx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old tx thread is existed\n");
+	}
+
+	if (!isp_ctx->composer_deinit_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread =
+			kthread_run(isp_composer_deinit_work, isp_ctx,
+				    "isp_composer_deinit_work");
+		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_deinit_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old rx thread is existed\n");
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	return 0;
+
+nomem:
+	mutex_unlock(&isp_ctx->lock);
+
+	return -ENOMEM;
+}
+
+void isp_composer_enqueue(struct device *dev,
+			  void *data,
+			  enum mtk_isp_scp_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+void isp_composer_hw_init(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
+	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_meta_config(struct device *dev,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct device *dev, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+
+	/* Wait for ISP_CMD_DEINIT command is handled done */
+	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
+				 msecs_to_jiffies(2000));
+	if (ret)
+		return;
+
+	dev_warn(dev, "Timeout & local de-init\n");
+	isp_composer_deinit(isp_ctx);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..fbd8593e9c2d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/types.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+int isp_composer_init(struct device *dev);
+void isp_composer_enqueue(struct device *dev, void *data,
+			  enum mtk_isp_scp_type type);
+void isp_composer_hw_init(struct device *dev);
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param);
+void isp_composer_hw_deinit(struct device *dev);
+void isp_composer_meta_config(struct device *dev, unsigned int dma);
+void isp_composer_stream(struct device *dev, int on);
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
 3 files changed, 579 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 30df10983f6a..95f0b1c8fa1c 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -3,5 +3,6 @@
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-scp.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..04519d0b942f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,371 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+
+	spin_lock(&isp_ctx->composer_txlist.lock);
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	spin_unlock(&isp_ctx->composer_txlist.lock);
+
+	mutex_lock(&isp_ctx->lock);
+	if (isp_ctx->composer_tx_thread.thread) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	if (isp_ctx->composer_deinit_thread.thread) {
+		wake_up(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread = NULL;
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM) ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_deinit_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
+				 kthread_should_stop());
+
+	dev_dbg(dev, "%s run deinit", __func__);
+	isp_composer_deinit(isp_ctx);
+
+	return 0;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK)
+		return;
+
+	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
+		dev_dbg(dev, "ack frame_num:%d",
+			ipi_msg->ack_info.frame_seq_no);
+		atomic_set(&isp_ctx->composed_frame_id,
+			   ipi_msg->ack_info.frame_seq_no);
+	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
+		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
+		atomic_set(&isp_ctx->scp_state, SCP_OFF);
+		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
+	}
+
+	atomic_dec_return(&isp_ctx->ipi_occupied);
+	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+}
+
+int isp_composer_init(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_ON);
+
+	mutex_lock(&isp_ctx->lock);
+	if (!isp_ctx->composer_tx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old tx thread is existed\n");
+	}
+
+	if (!isp_ctx->composer_deinit_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread =
+			kthread_run(isp_composer_deinit_work, isp_ctx,
+				    "isp_composer_deinit_work");
+		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_deinit_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old rx thread is existed\n");
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	return 0;
+
+nomem:
+	mutex_unlock(&isp_ctx->lock);
+
+	return -ENOMEM;
+}
+
+void isp_composer_enqueue(struct device *dev,
+			  void *data,
+			  enum mtk_isp_scp_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+void isp_composer_hw_init(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
+	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_meta_config(struct device *dev,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct device *dev, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+
+	/* Wait for ISP_CMD_DEINIT command is handled done */
+	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
+				 msecs_to_jiffies(2000));
+	if (ret)
+		return;
+
+	dev_warn(dev, "Timeout & local de-init\n");
+	isp_composer_deinit(isp_ctx);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..fbd8593e9c2d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/types.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+int isp_composer_init(struct device *dev);
+void isp_composer_enqueue(struct device *dev, void *data,
+			  enum mtk_isp_scp_type type);
+void isp_composer_hw_init(struct device *dev);
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param);
+void isp_composer_hw_deinit(struct device *dev);
+void isp_composer_meta_config(struct device *dev, unsigned int dma);
+void isp_composer_stream(struct device *dev, int on);
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds communication with the co-processor on the SoC
through the SCP driver. It supports bi-directional commands
to exchange data and perform command flow control function.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
 3 files changed, 579 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 30df10983f6a..95f0b1c8fa1c 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -3,5 +3,6 @@
 mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-scp.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
new file mode 100644
index 000000000000..04519d0b942f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
@@ -0,0 +1,371 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/kthread.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "mtk_cam.h"
+
+static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+
+	spin_lock(&isp_ctx->composer_txlist.lock);
+	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
+				 &isp_ctx->composer_txlist.queue,
+				 list_entry) {
+		list_del(&ipi_job->list_entry);
+		kfree(ipi_job);
+	}
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	spin_unlock(&isp_ctx->composer_txlist.lock);
+
+	mutex_lock(&isp_ctx->lock);
+	if (isp_ctx->composer_tx_thread.thread) {
+		kthread_stop(isp_ctx->composer_tx_thread.thread);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		isp_ctx->composer_tx_thread.thread = NULL;
+	}
+
+	if (isp_ctx->composer_deinit_thread.thread) {
+		wake_up(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread = NULL;
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	pm_runtime_put_sync(&p1_dev->pdev->dev);
+}
+
+/*
+ * Two kinds of flow control in isp_composer_tx_work.
+ *
+ * Case 1: IPI commands flow control. The maximum number of command queues is 3.
+ * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
+ * It is controlled by ipi_occupied.
+ * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
+ *
+ * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
+ * It is controlled by composing_frame.
+ * Frame buffer is sent by SCP_ISP_FRAME command.
+ */
+static int isp_composer_tx_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
+	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
+	int ret;
+
+	while (1) {
+		ret = wait_event_interruptible
+			(isp_ctx->composer_tx_thread.wq,
+			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM &&
+			 atomic_read(&isp_ctx->composing_frame)
+				< ISP_FRAME_COMPOSING_MAX_NUM) ||
+			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
+			 atomic_read(&isp_ctx->ipi_occupied)
+				< ISP_COMPOSING_MAX_NUM) ||
+			 kthread_should_stop());
+
+		if (kthread_should_stop())
+			break;
+
+		spin_lock(&composer_txlist->lock);
+		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
+			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
+						 &composer_txlist->queue,
+						 list_entry) {
+				if (isp_composer_work->type == SCP_ISP_CMD) {
+					dev_dbg(dev, "Found a cmd\n");
+					break;
+				}
+			}
+		} else {
+			if (atomic_read(&isp_ctx->composing_frame) >=
+				ISP_FRAME_COMPOSING_MAX_NUM) {
+				spin_unlock(&composer_txlist->lock);
+				continue;
+			}
+			isp_composer_work =
+			    list_first_entry_or_null
+				(&composer_txlist->queue,
+				 struct mtk_isp_queue_work,
+				 list_entry);
+		}
+
+		list_del(&isp_composer_work->list_entry);
+		atomic_dec(&composer_txlist->queue_cnt);
+		spin_unlock(&composer_txlist->lock);
+
+		if (isp_composer_work->type == SCP_ISP_CMD) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_CMD,
+				 &isp_composer_work->cmd,
+				 sizeof(isp_composer_work->cmd),
+				 0);
+			atomic_dec(&isp_ctx->cmd_queued);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			dev_dbg(dev,
+				"%s cmd id %d sent, %d ipi buf occupied",
+				__func__,
+				isp_composer_work->cmd.cmd_id,
+				atomic_read(&isp_ctx->ipi_occupied));
+		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
+			scp_ipi_send
+				(p1_dev->scp_pdev,
+				 SCP_IPI_ISP_FRAME,
+				 &isp_composer_work->frameparams,
+				 sizeof(isp_composer_work->frameparams),
+				 0);
+			atomic_inc(&isp_ctx->ipi_occupied);
+			atomic_inc(&isp_ctx->composing_frame);
+			dev_dbg(dev,
+				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
+				__func__,
+				isp_composer_work->frameparams.frame_seq_no,
+				atomic_read(&isp_ctx->ipi_occupied),
+				atomic_read(&isp_ctx->composing_frame));
+		} else {
+			dev_err(dev,
+				"ignore IPI type: %d!\n",
+				isp_composer_work->type);
+		}
+		kfree(isp_composer_work);
+	}
+	return ret;
+}
+
+static int isp_composer_deinit_work(void *data)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
+	struct device *dev = &p1_dev->pdev->dev;
+
+	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
+				 kthread_should_stop());
+
+	dev_dbg(dev, "%s run deinit", __func__);
+	isp_composer_deinit(isp_ctx);
+
+	return 0;
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK)
+		return;
+
+	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
+		dev_dbg(dev, "ack frame_num:%d",
+			ipi_msg->ack_info.frame_seq_no);
+		atomic_set(&isp_ctx->composed_frame_id,
+			   ipi_msg->ack_info.frame_seq_no);
+	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
+		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
+		atomic_set(&isp_ctx->scp_state, SCP_OFF);
+		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
+	}
+
+	atomic_dec_return(&isp_ctx->ipi_occupied);
+	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+}
+
+int isp_composer_init(struct device *dev)
+{
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev,
+			       SCP_IPI_ISP_CMD,
+			       isp_composer_handler,
+			       isp_ctx);
+	if (ret)
+		return ret;
+
+	atomic_set(&isp_ctx->cmd_queued, 0);
+	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
+	atomic_set(&isp_ctx->composing_frame, 0);
+	atomic_set(&isp_ctx->ipi_occupied, 0);
+	atomic_set(&isp_ctx->scp_state, SCP_ON);
+
+	mutex_lock(&isp_ctx->lock);
+	if (!isp_ctx->composer_tx_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
+		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
+		spin_lock_init(&isp_ctx->composer_txlist.lock);
+		isp_ctx->composer_tx_thread.thread =
+			kthread_run(isp_composer_tx_work, isp_ctx,
+				    "isp_composer_tx");
+		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_tx_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old tx thread is existed\n");
+	}
+
+	if (!isp_ctx->composer_deinit_thread.thread) {
+		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
+		isp_ctx->composer_deinit_thread.thread =
+			kthread_run(isp_composer_deinit_work, isp_ctx,
+				    "isp_composer_deinit_work");
+		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
+			dev_err(dev, "unable to start kthread\n");
+			isp_ctx->composer_deinit_thread.thread = NULL;
+			goto nomem;
+		}
+	} else {
+		dev_warn(dev, "old rx thread is existed\n");
+	}
+	mutex_unlock(&isp_ctx->lock);
+
+	return 0;
+
+nomem:
+	mutex_unlock(&isp_ctx->lock);
+
+	return -ENOMEM;
+}
+
+void isp_composer_enqueue(struct device *dev,
+			  void *data,
+			  enum mtk_isp_scp_type type)
+{
+	struct mtk_isp_queue_work *isp_composer_work;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
+	isp_composer_work->type = type;
+
+	switch (type) {
+	case SCP_ISP_CMD:
+		memcpy(&isp_composer_work->cmd, data,
+		       sizeof(isp_composer_work->cmd));
+		dev_dbg(dev, "Enq ipi cmd id:%d\n",
+			isp_composer_work->cmd.cmd_id);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		atomic_inc(&isp_ctx->cmd_queued);
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	case SCP_ISP_FRAME:
+		memcpy(&isp_composer_work->frameparams, data,
+		       sizeof(isp_composer_work->frameparams));
+		dev_dbg(dev, "Enq ipi frame_num:%d\n",
+			isp_composer_work->frameparams.frame_seq_no);
+
+		spin_lock(&isp_ctx->composer_txlist.lock);
+		list_add_tail(&isp_composer_work->list_entry,
+			      &isp_ctx->composer_txlist.queue);
+		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
+		spin_unlock(&isp_ctx->composer_txlist.lock);
+
+		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
+		break;
+	default:
+		break;
+	}
+}
+
+void isp_composer_hw_init(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
+	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
+	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_meta_config(struct device *dev,
+			      unsigned int dma)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
+	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_stream(struct device *dev, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+}
+
+void isp_composer_hw_deinit(struct device *dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct isp_p1_device *p1_dev = get_p1_device(dev);
+	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
+	int ret;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
+
+	/* Wait for ISP_CMD_DEINIT command is handled done */
+	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
+				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
+				 msecs_to_jiffies(2000));
+	if (ret)
+		return;
+
+	dev_warn(dev, "Timeout & local de-init\n");
+	isp_composer_deinit(isp_ctx);
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
new file mode 100644
index 000000000000..fbd8593e9c2d
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _MTK_ISP_SCP_H
+#define _MTK_ISP_SCP_H
+
+#include <linux/types.h>
+
+#include "mtk_cam-v4l2-util.h"
+
+/*
+ * struct img_size - image size information.
+ *
+ * @w: image width, the unit is pixel
+ * @h: image height, the unit is pixel
+ * @xsize: bytes per line based on width.
+ * @stride: bytes per line when changing line.
+ *          Normally, calculate new STRIDE based on
+ *          xsize + HW constrain(page or align).
+ *
+ */
+struct img_size {
+	__u32 w;
+	__u32 h;
+	__u32 xsize;
+	__u32 stride;
+} __packed;
+
+/*
+ * struct img_buffer - buffer address information.
+ *
+ * @iova: DMA address for external devices.
+ * @scp_addr: SCP address for external co-process unit.
+ *
+ */
+struct img_buffer {
+	__u32 iova;
+	__u32 scp_addr;
+} __packed;
+
+struct p1_img_crop {
+	__u32 left;
+	__u32 top;
+	__u32 width;
+	__u32 height;
+} __packed;
+
+struct p1_img_output {
+	struct img_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	__u8 pixel_byte;
+	__u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: indicate the sensor mode.
+ *              1: continuous
+ *              0: single
+ * @subsample: indicate to enables SOF subsample or not.
+ * @pixel_mode: describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: describe input data pattern.
+ * @raw_pixel_id: bayer sequence.
+ * @tg_fps: the fps rate of TG (time generator).
+ * @img_fmt: the image format of input source.
+ * @p1_img_crop: the crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	__u8 continuous;
+	__u8 subsample;
+	__u8 pixel_mode;
+	__u8 data_pattern;
+	__u8 raw_pixel_id;
+	__u16 tg_fps;
+	__u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - the image output parameters of main stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @pure_raw: indicate the image path control.
+ *            1: pure raw
+ *            0: processing raw
+ * @pure_raw_pack: indicate the image is packed or not.
+ *                 1: packed mode
+ *                 0: unpacked mode
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_main_out_param {
+	/* Bypass main out parameters */
+	__u8 bypass;
+	/* Control HW image raw path */
+	__u8 pure_raw;
+	/* Control HW image pack function */
+	__u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - the image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: indicate this device is enabled or disabled or not .
+ * @p1_img_output: the output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	/* Bypass resize parameters */
+	__u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_meta_out_param - output meta information.
+ *
+ * @enabled_meta_dmas: indicate which meta DMAs are enabled.
+ *
+ */
+struct cfg_meta_out_param {
+	__u32 enabled_meta_dmas;
+} __packed;
+
+struct p1_config_param {
+	/* Sensor/TG info */
+	struct cfg_in_param cfg_in_param;
+	/* IMGO DMA */
+	struct cfg_main_out_param cfg_main_param;
+	/* RRZO DMA */
+	struct cfg_resize_out_param cfg_resize_param;
+	/* 3A DMAs and other. */
+	struct cfg_meta_out_param cfg_meta_param;
+} __packed;
+
+struct p1_frame_param {
+	/* frame sequence number */
+	__u32 frame_seq_no;
+	/* SOF index */
+	__u32 sof_idx;
+	/* The memory address of tuning buffer from user space */
+	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+struct P1_meta_frame {
+	__u32 enabled_dma;
+	__u32 vb_index;
+	struct img_buffer meta_addr;
+} __packed;
+
+struct isp_init_info {
+	__u8 hw_module;
+	struct img_buffer cq_addr;
+} __packed;
+
+struct isp_ack_info {
+	__u8 cmd_id;
+	__u32 frame_seq_no;
+} __packed;
+
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_CONFIG_META,
+	ISP_CMD_ENQUEUE_META,
+	ISP_CMD_RESERVED,
+};
+
+struct mtk_isp_scp_p1_cmd {
+	__u8 cmd_id;
+	union {
+		struct isp_init_info frameparam;
+		struct p1_config_param config_param;
+		struct cfg_meta_out_param cfg_meta_out_param;
+		struct P1_meta_frame meta_frame;
+		__u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+enum mtk_isp_scp_type {
+	SCP_ISP_CMD = 0,
+	SCP_ISP_FRAME,
+};
+
+int isp_composer_init(struct device *dev);
+void isp_composer_enqueue(struct device *dev, void *data,
+			  enum mtk_isp_scp_type type);
+void isp_composer_hw_init(struct device *dev);
+void isp_composer_hw_config(struct device *dev,
+			    struct p1_config_param *config_param);
+void isp_composer_hw_deinit(struct device *dev);
+void isp_composer_meta_config(struct device *dev, unsigned int dma);
+void isp_composer_stream(struct device *dev, int on);
+
+#endif /* _MTK_ISP_SCP_H */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
  (?)
@ 2019-06-11  3:53     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
 3 files changed, 323 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 95f0b1c8fa1c..d545ca6f09c5 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
 mtk-cam-isp-objs += mtk_cam-scp.o
+mtk-cam-isp-objs += mtk_cam-smem.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
new file mode 100644
index 000000000000..a9845668ce10
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
@@ -0,0 +1,304 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <asm/cacheflush.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam-smem.h"
+
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+				    void *cpu_addr, dma_addr_t dma_addr,
+				    size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_setup_dma_ops(struct device *dev,
+				       struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
+	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
+	if (!smem_dev->smem_base || !smem_dev->smem_size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+		 __func__,
+		 smem_dev->dev,
+		 &smem_dev->smem_base,
+		 (smem_dev->smem_size / SZ_1M));
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+	if (ret) {
+		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
+		goto fail_map;
+	}
+
+	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+		 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int ret;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	if (ret)
+		return NULL;
+
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size;
+	dma_addr_t addr;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
+	if (!isp_ctx->scp_mem_pa || !size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
+
+	/* get iova address */
+	addr = dma_map_page_attrs(dev, phys_to_page(isp_ctx->scp_mem_pa), 0,
+				  size, DMA_BIDIRECTIONAL,
+				  DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		isp_ctx->scp_mem_pa = 0;
+		dev_err(dev, "Failed to map scp iova\n");
+		return -ENOMEM;
+	}
+
+	isp_ctx->scp_mem_iova = addr;
+
+	return 0;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+	int ret;
+
+	ret = mtk_cam_composer_dma_init(&p1_dev->isp_ctx);
+	if (ret)
+		return ret;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev.smem_dev = smem_dev->dev;
+
+	ret = mtk_cam_reserved_drm_sg_init(smem_dev);
+	if (ret)
+		return ret;
+
+	mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..981d47178e99
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, sean.cheng, sj.huang,
	frederic.chen, ryan.yu, rynn.wu, jungo.lin, frankie.chiu

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
 3 files changed, 323 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 95f0b1c8fa1c..d545ca6f09c5 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
 mtk-cam-isp-objs += mtk_cam-scp.o
+mtk-cam-isp-objs += mtk_cam-smem.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
new file mode 100644
index 000000000000..a9845668ce10
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
@@ -0,0 +1,304 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <asm/cacheflush.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam-smem.h"
+
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+				    void *cpu_addr, dma_addr_t dma_addr,
+				    size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_setup_dma_ops(struct device *dev,
+				       struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
+	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
+	if (!smem_dev->smem_base || !smem_dev->smem_size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+		 __func__,
+		 smem_dev->dev,
+		 &smem_dev->smem_base,
+		 (smem_dev->smem_size / SZ_1M));
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+	if (ret) {
+		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
+		goto fail_map;
+	}
+
+	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+		 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int ret;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	if (ret)
+		return NULL;
+
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size;
+	dma_addr_t addr;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
+	if (!isp_ctx->scp_mem_pa || !size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
+
+	/* get iova address */
+	addr = dma_map_page_attrs(dev, phys_to_page(isp_ctx->scp_mem_pa), 0,
+				  size, DMA_BIDIRECTIONAL,
+				  DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		isp_ctx->scp_mem_pa = 0;
+		dev_err(dev, "Failed to map scp iova\n");
+		return -ENOMEM;
+	}
+
+	isp_ctx->scp_mem_iova = addr;
+
+	return 0;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+	int ret;
+
+	ret = mtk_cam_composer_dma_init(&p1_dev->isp_ctx);
+	if (ret)
+		return ret;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev.smem_dev = smem_dev->dev;
+
+	ret = mtk_cam_reserved_drm_sg_init(smem_dev);
+	if (ret)
+		return ret;
+
+	mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..981d47178e99
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-06-11  3:53     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-06-11  3:53 UTC (permalink / raw)
  To: tfiga, hverkuil, laurent.pinchart, matthias.bgg, mchehab
  Cc: devicetree, sean.cheng, rynn.wu, srv_heupstream, robh, ryan.yu,
	frankie.chiu, jungo.lin, sj.huang, linux-mediatek, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

The purpose of this child device is to provide shared
memory management for exchanging tuning data between co-processor
and the Pass 1 unit of the camera ISP system, including cache
buffer handling.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/10972143/
---
 .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
 .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
 .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
 3 files changed, 323 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
index 95f0b1c8fa1c..d545ca6f09c5 100644
--- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
 mtk-cam-isp-objs += mtk_cam-v4l2-util.o
 mtk-cam-isp-objs += mtk_cam.o
 mtk-cam-isp-objs += mtk_cam-scp.o
+mtk-cam-isp-objs += mtk_cam-smem.o
 
 obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
new file mode 100644
index 000000000000..a9845668ce10
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
@@ -0,0 +1,304 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <asm/cacheflush.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam-smem.h"
+
+static struct dma_map_ops smem_dma_ops;
+
+struct mtk_cam_smem_dev {
+	struct device *dev;
+	struct sg_table sgt;
+	struct page **smem_pages;
+	dma_addr_t smem_base;
+	dma_addr_t smem_dma_base;
+	int smem_size;
+};
+
+struct dma_coherent_mem {
+	void		*virt_base;
+	dma_addr_t	device_base;
+	unsigned long	pfn_base;
+	int		size;
+	int		flags;
+	unsigned long	*bitmap;
+	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
+	bool		use_dev_dma_pfn_offset;
+};
+
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
+					 dma_addr_t iova)
+{
+	struct iommu_domain *domain;
+	dma_addr_t addr, limit;
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+
+	domain = iommu_get_domain_for_dev(dev);
+	if (!domain) {
+		dev_warn(dev, "No iommu group domain\n");
+		return 0;
+	}
+
+	addr = iommu_iova_to_phys(domain, iova);
+	limit = smem_dev->smem_base + smem_dev->smem_size;
+	if (addr < smem_dev->smem_base || addr >= limit) {
+		dev_err(dev,
+			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
+			&addr, &smem_dev->smem_base, &limit);
+		return 0;
+	}
+	return addr;
+}
+
+static int mtk_cam_smem_get_sgtable(struct device *dev,
+				    struct sg_table *sgt,
+				    void *cpu_addr, dma_addr_t dma_addr,
+				    size_t size, unsigned long attrs)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
+	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
+
+	dev_dbg(dev,
+		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
+		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
+
+	return sg_alloc_table_from_pages(sgt,
+		smem_dev->smem_pages + pages_start,
+		pages_count, 0, size, GFP_KERNEL);
+}
+
+static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
+				       dma_addr_t addr)
+{
+	struct device *dev = smem_dev->dev;
+	struct dma_coherent_mem *dma_mem = dev->dma_mem;
+
+	if (addr < smem_dev->smem_base ||
+	    addr > smem_dev->smem_base + smem_dev->smem_size) {
+		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
+		return NULL;
+	}
+	return dma_mem->virt_base + (addr - smem_dev->smem_base);
+}
+
+static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
+					 struct scatterlist *sgl, int nelems,
+					 enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_unmap_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
+					    struct scatterlist *sgl,
+					    int nelems,
+					    enum dma_data_direction dir)
+{
+	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
+	dma_addr_t scp_addr = sg_phys(sgl);
+	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
+
+	dev_dbg(dev,
+		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
+		&scp_addr, cpu_addr, sgl->length, dir);
+	__dma_map_area(cpu_addr, sgl->length, dir);
+}
+
+static void mtk_cam_smem_setup_dma_ops(struct device *dev,
+				       struct dma_map_ops *smem_ops)
+{
+	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
+	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
+	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
+	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
+	set_dma_ops(dev, smem_ops);
+}
+
+static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
+{
+	u32 size_align, n_pages;
+	struct device *dev = smem_dev->dev;
+	struct sg_table *sgt = &smem_dev->sgt;
+	struct page **pages;
+	dma_addr_t dma_addr;
+	unsigned int i;
+	int ret;
+
+	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
+	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
+	if (!smem_dev->smem_base || !smem_dev->smem_size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
+		 __func__,
+		 smem_dev->dev,
+		 &smem_dev->smem_base,
+		 (smem_dev->smem_size / SZ_1M));
+
+	size_align = PAGE_ALIGN(smem_dev->smem_size);
+	n_pages = size_align >> PAGE_SHIFT;
+
+	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return -ENOMEM;
+
+	for (i = 0; i < n_pages; i++)
+		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
+
+	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
+					size_align, GFP_KERNEL);
+	if (ret) {
+		dev_err(dev, "failed to alloca sg table:%d\n", ret);
+		goto fail_table_alloc;
+	}
+	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
+				      DMA_BIDIRECTIONAL,
+				      DMA_ATTR_SKIP_CPU_SYNC);
+	if (!sgt->nents) {
+		dev_err(dev, "failed to dma sg map\n");
+		goto fail_map;
+	}
+
+	dma_addr = sg_dma_address(sgt->sgl);
+	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
+					  dma_addr, size_align,
+					  DMA_MEMORY_EXCLUSIVE);
+	if (ret) {
+		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
+		goto fail_map;
+	}
+
+	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
+		 &smem_dev->smem_base, &dma_addr, size_align);
+
+	smem_dev->smem_size = size_align;
+	smem_dev->smem_pages = pages;
+	smem_dev->smem_dma_base = dma_addr;
+
+	return 0;
+
+fail_map:
+	sg_free_table(sgt);
+fail_table_alloc:
+	while (n_pages--)
+		__free_page(pages[n_pages]);
+	kfree(pages);
+
+	return -ENOMEM;
+}
+
+/* DMA memory related helper functions */
+static void mtk_cam_memdev_release(struct device *dev)
+{
+	vb2_dma_contig_clear_max_seg_size(dev);
+}
+
+static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
+					     const char *name)
+{
+	struct device *child;
+	int ret;
+
+	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+	if (!child)
+		return NULL;
+
+	child->parent = dev;
+	child->iommu_group = dev->iommu_group;
+	child->release = mtk_cam_memdev_release;
+	dev_set_name(child, name);
+	set_dma_ops(child, get_dma_ops(dev));
+	child->dma_mask = dev->dma_mask;
+	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
+	if (ret)
+		return NULL;
+
+	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
+
+	if (device_register(child)) {
+		device_del(child);
+		return NULL;
+	}
+
+	return child;
+}
+
+static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
+{
+	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
+	struct device *dev = &p1_dev->pdev->dev;
+	u32 size;
+	dma_addr_t addr;
+
+	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
+	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
+	if (!isp_ctx->scp_mem_pa || !size)
+		return -EPROBE_DEFER;
+
+	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
+
+	/* get iova address */
+	addr = dma_map_page_attrs(dev, phys_to_page(isp_ctx->scp_mem_pa), 0,
+				  size, DMA_BIDIRECTIONAL,
+				  DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		isp_ctx->scp_mem_pa = 0;
+		dev_err(dev, "Failed to map scp iova\n");
+		return -ENOMEM;
+	}
+
+	isp_ctx->scp_mem_iova = addr;
+
+	return 0;
+}
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev)
+{
+	struct device *dev = &p1_dev->pdev->dev;
+	struct mtk_cam_smem_dev *smem_dev;
+	int ret;
+
+	ret = mtk_cam_composer_dma_init(&p1_dev->isp_ctx);
+	if (ret)
+		return ret;
+
+	/* Allocate context */
+	smem_dev = devm_kzalloc(dev, sizeof(*smem_dev), GFP_KERNEL);
+	if (!smem_dev)
+		return -ENOMEM;
+
+	smem_dev->dev = mtk_cam_alloc_smem_dev(dev, "cam-smem");
+	if (!smem_dev->dev) {
+		dev_err(dev, "failed to alloc smem device\n");
+		return -ENODEV;
+	}
+	dev_set_drvdata(smem_dev->dev, smem_dev);
+	p1_dev->cam_dev.smem_dev = smem_dev->dev;
+
+	ret = mtk_cam_reserved_drm_sg_init(smem_dev);
+	if (ret)
+		return ret;
+
+	mtk_cam_smem_setup_dma_ops(smem_dev->dev, &smem_dma_ops);
+
+	return 0;
+}
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
new file mode 100644
index 000000000000..981d47178e99
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_ISP_SMEM_H
+#define __MTK_CAM_ISP_SMEM_H
+
+#include <linux/dma-mapping.h>
+
+#include "mtk_cam.h"
+
+int mtk_cam_reserved_memory_init(struct isp_p1_device *p1_dev);
+dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *smem_dev,
+					 dma_addr_t iova);
+
+#endif
+
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
  2019-06-11  3:53     ` Jungo Lin
  (?)
@ 2019-07-01  5:50       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  5:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> Reserved Mediatek ISP P1 V4L2 control number with 16.
> Moreover, add two V4L2 controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_MTK_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.

Could you explain with a bit more details what these binned width and height
would mean? How would they relate to the video CAPTURE node width and height?
Isn't this something that should be rather exposed as an appropriate
selection rectangle, instead of custom controls?

> 
> 2. V4L2_CID_MTK_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default value is 0 which outputs the pure raw bayer image
> from sesnor, without image processing in ISP HW.

Is it just effectively a full processing bypass? The driver seems to only
update the related configuration when the streaming starts. Can it be
controlled per-frame?

Generally this sounds more like something that should be modelled using the
media topology, similar to the example below.

/----------------\   /-------------------\   /--------------\
|                |---|                   |   |              |
| Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
|                |-\ |                   | | |              |
\----------------/ | \-------------------/ | \--------------/
                   |                       |
                   \-----------------------/

Then the userspace can select whether it wants the image from the capture
interface directly or procesed by the ISP by configuring the media links
appropriately.

The current limitation of this model is that it can't be easily configured
per-frame, as media configurations are not included in the requests yet.

[snip]

> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct v4l2_format *fmt;
> +
> +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> +
> +	if (is_width)
> +		ctrl->val = fmt->fmt.pix_mp.width;
> +	else
> +		ctrl->val = fmt->fmt.pix_mp.height;

This seems to contradict to what the comment in the header says, because it just
always returns the video node format and doesn't seem to care about whether
binning is enabled or not.

> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)

This is g_volatile_ctrl not, g_ctrl.

> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		handle_ctrl_get_process_raw(ctrl);
> +		break;

No need to provide getters for non-volatile controls. The
framework manages them.

> +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> +		handle_ctrl_get_bin_info(ctrl, 1);
> +		break;
> +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> +		handle_ctrl_get_bin_info(ctrl, 0);

It's trivial to get the value, so there isn't much benefit in having a
function to do so, especially if one needs something like a is_width
argument that further complicates the code.

> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		return handle_ctrl_set_process_raw(ctrl);

Same as above. The operation is too trivial to deserve a function.

> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> +	.name = "MTK CAM PROCESSING RAW",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> +	.name = "MTK CAM GET BIN WIDTH",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_WIDTH,
> +	.max = IMG_MAX_WIDTH,
> +	.step = 1,
> +	.def = IMG_MAX_WIDTH,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> +	.name = "MTK CAM GET BIN HEIGHT",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_HEIGHT,
> +	.max = IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {

This should be checked at the end, after all the controls are added.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-07-01  5:50       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  5:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> Reserved Mediatek ISP P1 V4L2 control number with 16.
> Moreover, add two V4L2 controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_MTK_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.

Could you explain with a bit more details what these binned width and height
would mean? How would they relate to the video CAPTURE node width and height?
Isn't this something that should be rather exposed as an appropriate
selection rectangle, instead of custom controls?

> 
> 2. V4L2_CID_MTK_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default value is 0 which outputs the pure raw bayer image
> from sesnor, without image processing in ISP HW.

Is it just effectively a full processing bypass? The driver seems to only
update the related configuration when the streaming starts. Can it be
controlled per-frame?

Generally this sounds more like something that should be modelled using the
media topology, similar to the example below.

/----------------\   /-------------------\   /--------------\
|                |---|                   |   |              |
| Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
|                |-\ |                   | | |              |
\----------------/ | \-------------------/ | \--------------/
                   |                       |
                   \-----------------------/

Then the userspace can select whether it wants the image from the capture
interface directly or procesed by the ISP by configuring the media links
appropriately.

The current limitation of this model is that it can't be easily configured
per-frame, as media configurations are not included in the requests yet.

[snip]

> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct v4l2_format *fmt;
> +
> +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> +
> +	if (is_width)
> +		ctrl->val = fmt->fmt.pix_mp.width;
> +	else
> +		ctrl->val = fmt->fmt.pix_mp.height;

This seems to contradict to what the comment in the header says, because it just
always returns the video node format and doesn't seem to care about whether
binning is enabled or not.

> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)

This is g_volatile_ctrl not, g_ctrl.

> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		handle_ctrl_get_process_raw(ctrl);
> +		break;

No need to provide getters for non-volatile controls. The
framework manages them.

> +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> +		handle_ctrl_get_bin_info(ctrl, 1);
> +		break;
> +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> +		handle_ctrl_get_bin_info(ctrl, 0);

It's trivial to get the value, so there isn't much benefit in having a
function to do so, especially if one needs something like a is_width
argument that further complicates the code.

> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		return handle_ctrl_set_process_raw(ctrl);

Same as above. The operation is too trivial to deserve a function.

> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> +	.name = "MTK CAM PROCESSING RAW",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> +	.name = "MTK CAM GET BIN WIDTH",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_WIDTH,
> +	.max = IMG_MAX_WIDTH,
> +	.step = 1,
> +	.def = IMG_MAX_WIDTH,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> +	.name = "MTK CAM GET BIN HEIGHT",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_HEIGHT,
> +	.max = IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {

This should be checked at the end, after all the controls are added.

Best regards,
Tomasz


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-07-01  5:50       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  5:50 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> Reserved Mediatek ISP P1 V4L2 control number with 16.
> Moreover, add two V4L2 controls for ISP P1 user space
> usage.
> 
> 1. V4L2_CID_MTK_GET_BIN_INFO
> - Provide the image output width & height in case
> camera binning mode is enabled.

Could you explain with a bit more details what these binned width and height
would mean? How would they relate to the video CAPTURE node width and height?
Isn't this something that should be rather exposed as an appropriate
selection rectangle, instead of custom controls?

> 
> 2. V4L2_CID_MTK_RAW_PATH
> - Export the path control of the main stream to user space.
> One is pure raw and the other is processing raw.
> The default value is 0 which outputs the pure raw bayer image
> from sesnor, without image processing in ISP HW.

Is it just effectively a full processing bypass? The driver seems to only
update the related configuration when the streaming starts. Can it be
controlled per-frame?

Generally this sounds more like something that should be modelled using the
media topology, similar to the example below.

/----------------\   /-------------------\   /--------------\
|                |---|                   |   |              |
| Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
|                |-\ |                   | | |              |
\----------------/ | \-------------------/ | \--------------/
                   |                       |
                   \-----------------------/

Then the userspace can select whether it wants the image from the capture
interface directly or procesed by the ISP by configuring the media links
appropriately.

The current limitation of this model is that it can't be easily configured
per-frame, as media configurations are not included in the requests yet.

[snip]

> +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct v4l2_format *fmt;
> +
> +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> +
> +	if (is_width)
> +		ctrl->val = fmt->fmt.pix_mp.width;
> +	else
> +		ctrl->val = fmt->fmt.pix_mp.height;

This seems to contradict to what the comment in the header says, because it just
always returns the video node format and doesn't seem to care about whether
binning is enabled or not.

> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> +
> +	return 0;
> +}
> +
> +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> +
> +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)

This is g_volatile_ctrl not, g_ctrl.

> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		handle_ctrl_get_process_raw(ctrl);
> +		break;

No need to provide getters for non-volatile controls. The
framework manages them.

> +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> +		handle_ctrl_get_bin_info(ctrl, 1);
> +		break;
> +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> +		handle_ctrl_get_bin_info(ctrl, 0);

It's trivial to get the value, so there isn't much benefit in having a
function to do so, especially if one needs something like a is_width
argument that further complicates the code.

> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	switch (ctrl->id) {
> +	case V4L2_CID_MTK_PROCESSING_RAW:
> +		return handle_ctrl_set_process_raw(ctrl);

Same as above. The operation is too trivial to deserve a function.

> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> +	.s_ctrl = mtk_cam_dev_s_ctrl,
> +};
> +
> +struct v4l2_ctrl_config mtk_cam_controls[] = {
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> +	.name = "MTK CAM PROCESSING RAW",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = 0,
> +	.max = 1,
> +	.step = 1,
> +	.def = 1,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> +	.name = "MTK CAM GET BIN WIDTH",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_WIDTH,
> +	.max = IMG_MAX_WIDTH,
> +	.step = 1,
> +	.def = IMG_MAX_WIDTH,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +	{
> +	.ops = &mtk_cam_dev_ctrl_ops,
> +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> +	.name = "MTK CAM GET BIN HEIGHT",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = IMG_MIN_HEIGHT,
> +	.max = IMG_MAX_HEIGHT,
> +	.step = 1,
> +	.def = IMG_MAX_HEIGHT,
> +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> +	},
> +};
> +
> +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> +		      struct v4l2_ctrl_handler *hdl)
> +{
> +	unsigned int i;
> +
> +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> +	if (hdl->error) {

This should be checked at the end, after all the controls are added.

Best regards,
Tomasz


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-06-11  3:53     ` [RFC,v3 " Jungo Lin
  (?)
@ 2019-07-01  7:25       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  7:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> The purpose of this child device is to provide shared
> memory management for exchanging tuning data between co-processor
> and the Pass 1 unit of the camera ISP system, including cache
> buffer handling.
> 

Looks like we haven't really progressed on getting this replaced with
something that doesn't require so much custom code. Let me propose something
better then.

We already have a reserved memory mode in DT. If it has a compatible string
of "shared-dma-pool", it would be registered in the coherent DMA framework
[1]. That would make it available for consumer devices to look-up.

Now if we add a "memory-region" property to the SCP device node and point it
to our reserved memory node, the SCP driver could look it up and hook to the
DMA mapping API using of_reserved_mem_device_init_by_idx[2].

That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
struct device use the coherent DMA ops, which operate on the assigned memory
pool. With that, the P1 driver could just directly use those calls to
manage the memory, without any custom code.

There is an example how this setup works in the s5p-mfc driver[3], but it
needs to be noted that it creates child nodes, because it can have more than
1 DMA port, which may need its own memory pool. In our case, we wouldn't
need child nodes and could just use the SCP device directly.

[1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
[2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
[3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075

Let me also post some specific comments below, in case we end up still
needing any of the code.

> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
>  3 files changed, 323 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> index 95f0b1c8fa1c..d545ca6f09c5 100644
> --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
>  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
>  mtk-cam-isp-objs += mtk_cam.o
>  mtk-cam-isp-objs += mtk_cam-scp.o
> +mtk-cam-isp-objs += mtk_cam-smem.o
>  
>  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> new file mode 100644
> index 000000000000..a9845668ce10
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> @@ -0,0 +1,304 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <asm/cacheflush.h>
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/iommu.h>
> +#include <linux/of.h>
> +#include <linux/of_fdt.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam-smem.h"
> +
> +static struct dma_map_ops smem_dma_ops;
> +
> +struct mtk_cam_smem_dev {
> +	struct device *dev;
> +	struct sg_table sgt;
> +	struct page **smem_pages;
> +	dma_addr_t smem_base;
> +	dma_addr_t smem_dma_base;
> +	int smem_size;
> +};
> +
> +struct dma_coherent_mem {
> +	void		*virt_base;
> +	dma_addr_t	device_base;
> +	unsigned long	pfn_base;
> +	int		size;
> +	int		flags;
> +	unsigned long	*bitmap;
> +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> +	bool		use_dev_dma_pfn_offset;
> +};
> +
> +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> +					 dma_addr_t iova)
> +{
> +	struct iommu_domain *domain;
> +	dma_addr_t addr, limit;
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +
> +	domain = iommu_get_domain_for_dev(dev);
> +	if (!domain) {
> +		dev_warn(dev, "No iommu group domain\n");
> +		return 0;
> +	}
> +
> +	addr = iommu_iova_to_phys(domain, iova);
> +	limit = smem_dev->smem_base + smem_dev->smem_size;
> +	if (addr < smem_dev->smem_base || addr >= limit) {
> +		dev_err(dev,
> +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> +			&addr, &smem_dev->smem_base, &limit);
> +		return 0;
> +	}
> +	return addr;
> +}

This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
and then the address wouldn't be valid, because it would point outside of
the address range allowed for SCP to access and also it would only point to
the first page backing the IOVA.

The correct approach would be to always carry SCP DMA address and IOVA
together in some kind of struct describing such buffers.

> +
> +static int mtk_cam_smem_get_sgtable(struct device *dev,
> +				    struct sg_table *sgt,
> +				    void *cpu_addr, dma_addr_t dma_addr,
> +				    size_t size, unsigned long attrs)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> +
> +	dev_dbg(dev,
> +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> +
> +	return sg_alloc_table_from_pages(sgt,
> +		smem_dev->smem_pages + pages_start,
> +		pages_count, 0, size, GFP_KERNEL);
> +}

This should be just dma_get_sgtable_attrs(), in the approach I suggested at
the top.

> +
> +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> +				       dma_addr_t addr)
> +{
> +	struct device *dev = smem_dev->dev;
> +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> +
> +	if (addr < smem_dev->smem_base ||
> +	    addr > smem_dev->smem_base + smem_dev->smem_size) {

This is off by one, should be >=.

Also, this wouldn't really guarantee the CPU access the caller is going to
do is valid, because it doesn't consider the access operation size.

Generally I'd suggest designing the code so that it doesn't have to convert
offset addresses between different address spaces.

> +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> +		return NULL;
> +	}
> +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> +}
> +
> +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> +					 struct scatterlist *sgl, int nelems,
> +					 enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_unmap_area(cpu_addr, sgl->length, dir);

It's not allowed to use this function anywhere outside of the DMA API
internals. See the comment [4].

[4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112

> +}
> +
> +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> +					    struct scatterlist *sgl,
> +					    int nelems,
> +					    enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_map_area(cpu_addr, sgl->length, dir);

Ditto.

> +}
> +
> +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> +				       struct dma_map_ops *smem_ops)
> +{
> +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> +	set_dma_ops(dev, smem_ops);
> +}
> +
> +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> +{
> +	u32 size_align, n_pages;
> +	struct device *dev = smem_dev->dev;
> +	struct sg_table *sgt = &smem_dev->sgt;
> +	struct page **pages;
> +	dma_addr_t dma_addr;
> +	unsigned int i;
> +	int ret;
> +
> +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> +		 __func__,
> +		 smem_dev->dev,
> +		 &smem_dev->smem_base,
> +		 (smem_dev->smem_size / SZ_1M));
> +
> +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> +	n_pages = size_align >> PAGE_SHIFT;
> +
> +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> +	if (!pages)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < n_pages; i++)
> +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> +
> +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> +					size_align, GFP_KERNEL);
> +	if (ret) {
> +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> +		goto fail_table_alloc;
> +	}
> +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> +				      DMA_BIDIRECTIONAL,
> +				      DMA_ATTR_SKIP_CPU_SYNC);
> +	if (!sgt->nents) {
> +		dev_err(dev, "failed to dma sg map\n");
> +		goto fail_map;
> +	}
> +
> +	dma_addr = sg_dma_address(sgt->sgl);
> +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> +					  dma_addr, size_align,
> +					  DMA_MEMORY_EXCLUSIVE);
> +	if (ret) {
> +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> +		goto fail_map;
> +	}
> +
> +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> +		 &smem_dev->smem_base, &dma_addr, size_align);
> +
> +	smem_dev->smem_size = size_align;
> +	smem_dev->smem_pages = pages;
> +	smem_dev->smem_dma_base = dma_addr;
> +
> +	return 0;
> +
> +fail_map:
> +	sg_free_table(sgt);
> +fail_table_alloc:
> +	while (n_pages--)
> +		__free_page(pages[n_pages]);
> +	kfree(pages);
> +
> +	return -ENOMEM;
> +}
> +
> +/* DMA memory related helper functions */
> +static void mtk_cam_memdev_release(struct device *dev)
> +{
> +	vb2_dma_contig_clear_max_seg_size(dev);
> +}
> +
> +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> +					     const char *name)
> +{
> +	struct device *child;
> +	int ret;
> +
> +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> +	if (!child)
> +		return NULL;
> +
> +	child->parent = dev;
> +	child->iommu_group = dev->iommu_group;

This isn't something that can be set explicitly. It's an internal field of
the IOMMU subsystem.

> +	child->release = mtk_cam_memdev_release;
> +	dev_set_name(child, name);
> +	set_dma_ops(child, get_dma_ops(dev));
> +	child->dma_mask = dev->dma_mask;
> +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> +	if (ret)
> +		return NULL;
> +
> +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> +
> +	if (device_register(child)) {
> +		device_del(child);
> +		return NULL;
> +	}
> +
> +	return child;
> +}

We shouldn't need child devices, just one SCP device, as I mentioned above.

> +
> +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	u32 size;
> +	dma_addr_t addr;
> +
> +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> +	if (!isp_ctx->scp_mem_pa || !size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);

This isn't something that deserves the "info" log level. Should be "dbg"
or removed.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-01  7:25       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  7:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> The purpose of this child device is to provide shared
> memory management for exchanging tuning data between co-processor
> and the Pass 1 unit of the camera ISP system, including cache
> buffer handling.
> 

Looks like we haven't really progressed on getting this replaced with
something that doesn't require so much custom code. Let me propose something
better then.

We already have a reserved memory mode in DT. If it has a compatible string
of "shared-dma-pool", it would be registered in the coherent DMA framework
[1]. That would make it available for consumer devices to look-up.

Now if we add a "memory-region" property to the SCP device node and point it
to our reserved memory node, the SCP driver could look it up and hook to the
DMA mapping API using of_reserved_mem_device_init_by_idx[2].

That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
struct device use the coherent DMA ops, which operate on the assigned memory
pool. With that, the P1 driver could just directly use those calls to
manage the memory, without any custom code.

There is an example how this setup works in the s5p-mfc driver[3], but it
needs to be noted that it creates child nodes, because it can have more than
1 DMA port, which may need its own memory pool. In our case, we wouldn't
need child nodes and could just use the SCP device directly.

[1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
[2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
[3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075

Let me also post some specific comments below, in case we end up still
needing any of the code.

> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
>  3 files changed, 323 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> index 95f0b1c8fa1c..d545ca6f09c5 100644
> --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
>  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
>  mtk-cam-isp-objs += mtk_cam.o
>  mtk-cam-isp-objs += mtk_cam-scp.o
> +mtk-cam-isp-objs += mtk_cam-smem.o
>  
>  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> new file mode 100644
> index 000000000000..a9845668ce10
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> @@ -0,0 +1,304 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <asm/cacheflush.h>
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/iommu.h>
> +#include <linux/of.h>
> +#include <linux/of_fdt.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam-smem.h"
> +
> +static struct dma_map_ops smem_dma_ops;
> +
> +struct mtk_cam_smem_dev {
> +	struct device *dev;
> +	struct sg_table sgt;
> +	struct page **smem_pages;
> +	dma_addr_t smem_base;
> +	dma_addr_t smem_dma_base;
> +	int smem_size;
> +};
> +
> +struct dma_coherent_mem {
> +	void		*virt_base;
> +	dma_addr_t	device_base;
> +	unsigned long	pfn_base;
> +	int		size;
> +	int		flags;
> +	unsigned long	*bitmap;
> +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> +	bool		use_dev_dma_pfn_offset;
> +};
> +
> +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> +					 dma_addr_t iova)
> +{
> +	struct iommu_domain *domain;
> +	dma_addr_t addr, limit;
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +
> +	domain = iommu_get_domain_for_dev(dev);
> +	if (!domain) {
> +		dev_warn(dev, "No iommu group domain\n");
> +		return 0;
> +	}
> +
> +	addr = iommu_iova_to_phys(domain, iova);
> +	limit = smem_dev->smem_base + smem_dev->smem_size;
> +	if (addr < smem_dev->smem_base || addr >= limit) {
> +		dev_err(dev,
> +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> +			&addr, &smem_dev->smem_base, &limit);
> +		return 0;
> +	}
> +	return addr;
> +}

This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
and then the address wouldn't be valid, because it would point outside of
the address range allowed for SCP to access and also it would only point to
the first page backing the IOVA.

The correct approach would be to always carry SCP DMA address and IOVA
together in some kind of struct describing such buffers.

> +
> +static int mtk_cam_smem_get_sgtable(struct device *dev,
> +				    struct sg_table *sgt,
> +				    void *cpu_addr, dma_addr_t dma_addr,
> +				    size_t size, unsigned long attrs)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> +
> +	dev_dbg(dev,
> +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> +
> +	return sg_alloc_table_from_pages(sgt,
> +		smem_dev->smem_pages + pages_start,
> +		pages_count, 0, size, GFP_KERNEL);
> +}

This should be just dma_get_sgtable_attrs(), in the approach I suggested at
the top.

> +
> +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> +				       dma_addr_t addr)
> +{
> +	struct device *dev = smem_dev->dev;
> +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> +
> +	if (addr < smem_dev->smem_base ||
> +	    addr > smem_dev->smem_base + smem_dev->smem_size) {

This is off by one, should be >=.

Also, this wouldn't really guarantee the CPU access the caller is going to
do is valid, because it doesn't consider the access operation size.

Generally I'd suggest designing the code so that it doesn't have to convert
offset addresses between different address spaces.

> +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> +		return NULL;
> +	}
> +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> +}
> +
> +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> +					 struct scatterlist *sgl, int nelems,
> +					 enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_unmap_area(cpu_addr, sgl->length, dir);

It's not allowed to use this function anywhere outside of the DMA API
internals. See the comment [4].

[4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112

> +}
> +
> +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> +					    struct scatterlist *sgl,
> +					    int nelems,
> +					    enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_map_area(cpu_addr, sgl->length, dir);

Ditto.

> +}
> +
> +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> +				       struct dma_map_ops *smem_ops)
> +{
> +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> +	set_dma_ops(dev, smem_ops);
> +}
> +
> +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> +{
> +	u32 size_align, n_pages;
> +	struct device *dev = smem_dev->dev;
> +	struct sg_table *sgt = &smem_dev->sgt;
> +	struct page **pages;
> +	dma_addr_t dma_addr;
> +	unsigned int i;
> +	int ret;
> +
> +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> +		 __func__,
> +		 smem_dev->dev,
> +		 &smem_dev->smem_base,
> +		 (smem_dev->smem_size / SZ_1M));
> +
> +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> +	n_pages = size_align >> PAGE_SHIFT;
> +
> +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> +	if (!pages)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < n_pages; i++)
> +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> +
> +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> +					size_align, GFP_KERNEL);
> +	if (ret) {
> +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> +		goto fail_table_alloc;
> +	}
> +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> +				      DMA_BIDIRECTIONAL,
> +				      DMA_ATTR_SKIP_CPU_SYNC);
> +	if (!sgt->nents) {
> +		dev_err(dev, "failed to dma sg map\n");
> +		goto fail_map;
> +	}
> +
> +	dma_addr = sg_dma_address(sgt->sgl);
> +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> +					  dma_addr, size_align,
> +					  DMA_MEMORY_EXCLUSIVE);
> +	if (ret) {
> +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> +		goto fail_map;
> +	}
> +
> +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> +		 &smem_dev->smem_base, &dma_addr, size_align);
> +
> +	smem_dev->smem_size = size_align;
> +	smem_dev->smem_pages = pages;
> +	smem_dev->smem_dma_base = dma_addr;
> +
> +	return 0;
> +
> +fail_map:
> +	sg_free_table(sgt);
> +fail_table_alloc:
> +	while (n_pages--)
> +		__free_page(pages[n_pages]);
> +	kfree(pages);
> +
> +	return -ENOMEM;
> +}
> +
> +/* DMA memory related helper functions */
> +static void mtk_cam_memdev_release(struct device *dev)
> +{
> +	vb2_dma_contig_clear_max_seg_size(dev);
> +}
> +
> +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> +					     const char *name)
> +{
> +	struct device *child;
> +	int ret;
> +
> +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> +	if (!child)
> +		return NULL;
> +
> +	child->parent = dev;
> +	child->iommu_group = dev->iommu_group;

This isn't something that can be set explicitly. It's an internal field of
the IOMMU subsystem.

> +	child->release = mtk_cam_memdev_release;
> +	dev_set_name(child, name);
> +	set_dma_ops(child, get_dma_ops(dev));
> +	child->dma_mask = dev->dma_mask;
> +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> +	if (ret)
> +		return NULL;
> +
> +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> +
> +	if (device_register(child)) {
> +		device_del(child);
> +		return NULL;
> +	}
> +
> +	return child;
> +}

We shouldn't need child devices, just one SCP device, as I mentioned above.

> +
> +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	u32 size;
> +	dma_addr_t addr;
> +
> +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> +	if (!isp_ctx->scp_mem_pa || !size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);

This isn't something that deserves the "info" log level. Should be "dbg"
or removed.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-01  7:25       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-01  7:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> The purpose of this child device is to provide shared
> memory management for exchanging tuning data between co-processor
> and the Pass 1 unit of the camera ISP system, including cache
> buffer handling.
> 

Looks like we haven't really progressed on getting this replaced with
something that doesn't require so much custom code. Let me propose something
better then.

We already have a reserved memory mode in DT. If it has a compatible string
of "shared-dma-pool", it would be registered in the coherent DMA framework
[1]. That would make it available for consumer devices to look-up.

Now if we add a "memory-region" property to the SCP device node and point it
to our reserved memory node, the SCP driver could look it up and hook to the
DMA mapping API using of_reserved_mem_device_init_by_idx[2].

That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
struct device use the coherent DMA ops, which operate on the assigned memory
pool. With that, the P1 driver could just directly use those calls to
manage the memory, without any custom code.

There is an example how this setup works in the s5p-mfc driver[3], but it
needs to be noted that it creates child nodes, because it can have more than
1 DMA port, which may need its own memory pool. In our case, we wouldn't
need child nodes and could just use the SCP device directly.

[1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
[2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
[3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075

Let me also post some specific comments below, in case we end up still
needing any of the code.

> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
>  3 files changed, 323 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> 
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> index 95f0b1c8fa1c..d545ca6f09c5 100644
> --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
>  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
>  mtk-cam-isp-objs += mtk_cam.o
>  mtk-cam-isp-objs += mtk_cam-scp.o
> +mtk-cam-isp-objs += mtk_cam-smem.o
>  
>  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> new file mode 100644
> index 000000000000..a9845668ce10
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> @@ -0,0 +1,304 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <asm/cacheflush.h>
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/iommu.h>
> +#include <linux/of.h>
> +#include <linux/of_fdt.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam-smem.h"
> +
> +static struct dma_map_ops smem_dma_ops;
> +
> +struct mtk_cam_smem_dev {
> +	struct device *dev;
> +	struct sg_table sgt;
> +	struct page **smem_pages;
> +	dma_addr_t smem_base;
> +	dma_addr_t smem_dma_base;
> +	int smem_size;
> +};
> +
> +struct dma_coherent_mem {
> +	void		*virt_base;
> +	dma_addr_t	device_base;
> +	unsigned long	pfn_base;
> +	int		size;
> +	int		flags;
> +	unsigned long	*bitmap;
> +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> +	bool		use_dev_dma_pfn_offset;
> +};
> +
> +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> +					 dma_addr_t iova)
> +{
> +	struct iommu_domain *domain;
> +	dma_addr_t addr, limit;
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +
> +	domain = iommu_get_domain_for_dev(dev);
> +	if (!domain) {
> +		dev_warn(dev, "No iommu group domain\n");
> +		return 0;
> +	}
> +
> +	addr = iommu_iova_to_phys(domain, iova);
> +	limit = smem_dev->smem_base + smem_dev->smem_size;
> +	if (addr < smem_dev->smem_base || addr >= limit) {
> +		dev_err(dev,
> +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> +			&addr, &smem_dev->smem_base, &limit);
> +		return 0;
> +	}
> +	return addr;
> +}

This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
and then the address wouldn't be valid, because it would point outside of
the address range allowed for SCP to access and also it would only point to
the first page backing the IOVA.

The correct approach would be to always carry SCP DMA address and IOVA
together in some kind of struct describing such buffers.

> +
> +static int mtk_cam_smem_get_sgtable(struct device *dev,
> +				    struct sg_table *sgt,
> +				    void *cpu_addr, dma_addr_t dma_addr,
> +				    size_t size, unsigned long attrs)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> +
> +	dev_dbg(dev,
> +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> +
> +	return sg_alloc_table_from_pages(sgt,
> +		smem_dev->smem_pages + pages_start,
> +		pages_count, 0, size, GFP_KERNEL);
> +}

This should be just dma_get_sgtable_attrs(), in the approach I suggested at
the top.

> +
> +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> +				       dma_addr_t addr)
> +{
> +	struct device *dev = smem_dev->dev;
> +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> +
> +	if (addr < smem_dev->smem_base ||
> +	    addr > smem_dev->smem_base + smem_dev->smem_size) {

This is off by one, should be >=.

Also, this wouldn't really guarantee the CPU access the caller is going to
do is valid, because it doesn't consider the access operation size.

Generally I'd suggest designing the code so that it doesn't have to convert
offset addresses between different address spaces.

> +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> +		return NULL;
> +	}
> +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> +}
> +
> +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> +					 struct scatterlist *sgl, int nelems,
> +					 enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_unmap_area(cpu_addr, sgl->length, dir);

It's not allowed to use this function anywhere outside of the DMA API
internals. See the comment [4].

[4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112

> +}
> +
> +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> +					    struct scatterlist *sgl,
> +					    int nelems,
> +					    enum dma_data_direction dir)
> +{
> +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> +	dma_addr_t scp_addr = sg_phys(sgl);
> +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> +
> +	dev_dbg(dev,
> +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> +		&scp_addr, cpu_addr, sgl->length, dir);
> +	__dma_map_area(cpu_addr, sgl->length, dir);

Ditto.

> +}
> +
> +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> +				       struct dma_map_ops *smem_ops)
> +{
> +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> +	set_dma_ops(dev, smem_ops);
> +}
> +
> +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> +{
> +	u32 size_align, n_pages;
> +	struct device *dev = smem_dev->dev;
> +	struct sg_table *sgt = &smem_dev->sgt;
> +	struct page **pages;
> +	dma_addr_t dma_addr;
> +	unsigned int i;
> +	int ret;
> +
> +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> +		 __func__,
> +		 smem_dev->dev,
> +		 &smem_dev->smem_base,
> +		 (smem_dev->smem_size / SZ_1M));
> +
> +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> +	n_pages = size_align >> PAGE_SHIFT;
> +
> +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> +	if (!pages)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < n_pages; i++)
> +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> +
> +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> +					size_align, GFP_KERNEL);
> +	if (ret) {
> +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> +		goto fail_table_alloc;
> +	}
> +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> +				      DMA_BIDIRECTIONAL,
> +				      DMA_ATTR_SKIP_CPU_SYNC);
> +	if (!sgt->nents) {
> +		dev_err(dev, "failed to dma sg map\n");
> +		goto fail_map;
> +	}
> +
> +	dma_addr = sg_dma_address(sgt->sgl);
> +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> +					  dma_addr, size_align,
> +					  DMA_MEMORY_EXCLUSIVE);
> +	if (ret) {
> +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> +		goto fail_map;
> +	}
> +
> +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> +		 &smem_dev->smem_base, &dma_addr, size_align);
> +
> +	smem_dev->smem_size = size_align;
> +	smem_dev->smem_pages = pages;
> +	smem_dev->smem_dma_base = dma_addr;
> +
> +	return 0;
> +
> +fail_map:
> +	sg_free_table(sgt);
> +fail_table_alloc:
> +	while (n_pages--)
> +		__free_page(pages[n_pages]);
> +	kfree(pages);
> +
> +	return -ENOMEM;
> +}
> +
> +/* DMA memory related helper functions */
> +static void mtk_cam_memdev_release(struct device *dev)
> +{
> +	vb2_dma_contig_clear_max_seg_size(dev);
> +}
> +
> +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> +					     const char *name)
> +{
> +	struct device *child;
> +	int ret;
> +
> +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> +	if (!child)
> +		return NULL;
> +
> +	child->parent = dev;
> +	child->iommu_group = dev->iommu_group;

This isn't something that can be set explicitly. It's an internal field of
the IOMMU subsystem.

> +	child->release = mtk_cam_memdev_release;
> +	dev_set_name(child, name);
> +	set_dma_ops(child, get_dma_ops(dev));
> +	child->dma_mask = dev->dma_mask;
> +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> +	if (ret)
> +		return NULL;
> +
> +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> +
> +	if (device_register(child)) {
> +		device_del(child);
> +		return NULL;
> +	}
> +
> +	return child;
> +}

We shouldn't need child devices, just one SCP device, as I mentioned above.

> +
> +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	u32 size;
> +	dma_addr_t addr;
> +
> +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> +	if (!isp_ctx->scp_mem_pa || !size)
> +		return -EPROBE_DEFER;
> +
> +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);

This isn't something that deserves the "info" log level. Should be "dbg"
or removed.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
  2019-07-01  5:50       ` Tomasz Figa
  (?)
@ 2019-07-02 11:34         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-02 11:34 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Tomasz,

On Mon, 2019-07-01 at 14:50 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 V4L2 control number with 16.
> > Moreover, add two V4L2 controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_MTK_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> 
> Could you explain with a bit more details what these binned width and height
> would mean? How would they relate to the video CAPTURE node width and height?
> Isn't this something that should be rather exposed as an appropriate
> selection rectangle, instead of custom controls?
> 

Frontal binning is Mediatek ISP P1 HW function and it is designed for
low power saving. The input sensor resolution will be divided by 2
including width & height in the source side of ISP HW and lower the
overall ISP throughput. So the ISP clock could be running in the low
speed for power saving. However, the image quality will be bad in this
mode. If the frontal binning is enabled by ISP driver, user space 3A
algorithm needs to know the new resolution of TG. Btw, current ISP
driver doesn't support this function. We just export this V4L2 contorl
for future support. But, after internal discussing, we will remove this
v4l2 control in next patch.

> > 
> > 2. V4L2_CID_MTK_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default value is 0 which outputs the pure raw bayer image
> > from sesnor, without image processing in ISP HW.
> 
> Is it just effectively a full processing bypass? The driver seems to only
> update the related configuration when the streaming starts. Can it be
> controlled per-frame?
> 
> Generally this sounds more like something that should be modelled using the
> media topology, similar to the example below.
> 
> /----------------\   /-------------------\   /--------------\
> |                |---|                   |   |              |
> | Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
> |                |-\ |                   | | |              |
> \----------------/ | \-------------------/ | \--------------/
>                    |                       |
>                    \-----------------------/
> 
> Then the userspace can select whether it wants the image from the capture
> interface directly or procesed by the ISP by configuring the media links
> appropriately.
> 
> The current limitation of this model is that it can't be easily configured
> per-frame, as media configurations are not included in the requests yet.
> 
> [snip]
> 

For this V4L2 control, it is designed for per stream, not per frame and
just implemented for future usage. As the same conclusion with the above
one, we will remove this, either in current version. If we have strong
requirement on this, we could adopt your suggestion to use media
topology to per stream control.


> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct v4l2_format *fmt;
> > +
> > +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> > +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> > +
> > +	if (is_width)
> > +		ctrl->val = fmt->fmt.pix_mp.width;
> > +	else
> > +		ctrl->val = fmt->fmt.pix_mp.height;
> 
> This seems to contradict to what the comment in the header says, because it just
> always returns the video node format and doesn't seem to care about whether
> binning is enabled or not.
> 

This is because binning mode is not supported in current driver's
implementation and no request on this function. We just create this for
future usage. In order to avoid confusion, we will remove this.

> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> > +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> 
> This is g_volatile_ctrl not, g_ctrl.
> 

Ok, thanks for your correction.

> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		handle_ctrl_get_process_raw(ctrl);
> > +		break;
> 
> No need to provide getters for non-volatile controls. The
> framework manages them.
> 

Ok, thanks for your correction.

> > +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> > +		handle_ctrl_get_bin_info(ctrl, 1);
> > +		break;
> > +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> > +		handle_ctrl_get_bin_info(ctrl, 0);
> 
> It's trivial to get the value, so there isn't much benefit in having a
> function to do so, especially if one needs something like a is_width
> argument that further complicates the code.
> 

Ok, we will pay attention in this of implementation next time.

> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		return handle_ctrl_set_process_raw(ctrl);
> 
> Same as above. The operation is too trivial to deserve a function.
> 

Ok, got it.

> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> > +	.name = "MTK CAM PROCESSING RAW",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> > +	.name = "MTK CAM GET BIN WIDTH",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_WIDTH,
> > +	.max = IMG_MAX_WIDTH,
> > +	.step = 1,
> > +	.def = IMG_MAX_WIDTH,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> > +	.name = "MTK CAM GET BIN HEIGHT",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_HEIGHT,
> > +	.max = IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> 
> This should be checked at the end, after all the controls are added.
> 

Ok, got it.

> Best regards,
> Tomasz
> 

Based on above comments, we will remove mtk_cam_ctrl.h & mtk_cam_ctrl.c
in next patch to avoid over design.

Thanks for your comments.

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-07-02 11:34         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-02 11:34 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Tomasz,

On Mon, 2019-07-01 at 14:50 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 V4L2 control number with 16.
> > Moreover, add two V4L2 controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_MTK_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> 
> Could you explain with a bit more details what these binned width and height
> would mean? How would they relate to the video CAPTURE node width and height?
> Isn't this something that should be rather exposed as an appropriate
> selection rectangle, instead of custom controls?
> 

Frontal binning is Mediatek ISP P1 HW function and it is designed for
low power saving. The input sensor resolution will be divided by 2
including width & height in the source side of ISP HW and lower the
overall ISP throughput. So the ISP clock could be running in the low
speed for power saving. However, the image quality will be bad in this
mode. If the frontal binning is enabled by ISP driver, user space 3A
algorithm needs to know the new resolution of TG. Btw, current ISP
driver doesn't support this function. We just export this V4L2 contorl
for future support. But, after internal discussing, we will remove this
v4l2 control in next patch.

> > 
> > 2. V4L2_CID_MTK_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default value is 0 which outputs the pure raw bayer image
> > from sesnor, without image processing in ISP HW.
> 
> Is it just effectively a full processing bypass? The driver seems to only
> update the related configuration when the streaming starts. Can it be
> controlled per-frame?
> 
> Generally this sounds more like something that should be modelled using the
> media topology, similar to the example below.
> 
> /----------------\   /-------------------\   /--------------\
> |                |---|                   |   |              |
> | Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
> |                |-\ |                   | | |              |
> \----------------/ | \-------------------/ | \--------------/
>                    |                       |
>                    \-----------------------/
> 
> Then the userspace can select whether it wants the image from the capture
> interface directly or procesed by the ISP by configuring the media links
> appropriately.
> 
> The current limitation of this model is that it can't be easily configured
> per-frame, as media configurations are not included in the requests yet.
> 
> [snip]
> 

For this V4L2 control, it is designed for per stream, not per frame and
just implemented for future usage. As the same conclusion with the above
one, we will remove this, either in current version. If we have strong
requirement on this, we could adopt your suggestion to use media
topology to per stream control.


> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct v4l2_format *fmt;
> > +
> > +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> > +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> > +
> > +	if (is_width)
> > +		ctrl->val = fmt->fmt.pix_mp.width;
> > +	else
> > +		ctrl->val = fmt->fmt.pix_mp.height;
> 
> This seems to contradict to what the comment in the header says, because it just
> always returns the video node format and doesn't seem to care about whether
> binning is enabled or not.
> 

This is because binning mode is not supported in current driver's
implementation and no request on this function. We just create this for
future usage. In order to avoid confusion, we will remove this.

> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> > +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> 
> This is g_volatile_ctrl not, g_ctrl.
> 

Ok, thanks for your correction.

> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		handle_ctrl_get_process_raw(ctrl);
> > +		break;
> 
> No need to provide getters for non-volatile controls. The
> framework manages them.
> 

Ok, thanks for your correction.

> > +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> > +		handle_ctrl_get_bin_info(ctrl, 1);
> > +		break;
> > +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> > +		handle_ctrl_get_bin_info(ctrl, 0);
> 
> It's trivial to get the value, so there isn't much benefit in having a
> function to do so, especially if one needs something like a is_width
> argument that further complicates the code.
> 

Ok, we will pay attention in this of implementation next time.

> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		return handle_ctrl_set_process_raw(ctrl);
> 
> Same as above. The operation is too trivial to deserve a function.
> 

Ok, got it.

> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> > +	.name = "MTK CAM PROCESSING RAW",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> > +	.name = "MTK CAM GET BIN WIDTH",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_WIDTH,
> > +	.max = IMG_MAX_WIDTH,
> > +	.step = 1,
> > +	.def = IMG_MAX_WIDTH,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> > +	.name = "MTK CAM GET BIN HEIGHT",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_HEIGHT,
> > +	.max = IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> 
> This should be checked at the end, after all the controls are added.
> 

Ok, got it.

> Best regards,
> Tomasz
> 

Based on above comments, we will remove mtk_cam_ctrl.h & mtk_cam_ctrl.c
in next patch to avoid over design.

Thanks for your comments.

Jungo



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control
@ 2019-07-02 11:34         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-02 11:34 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Tomasz,

On Mon, 2019-07-01 at 14:50 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:40AM +0800, Jungo Lin wrote:
> > Reserved Mediatek ISP P1 V4L2 control number with 16.
> > Moreover, add two V4L2 controls for ISP P1 user space
> > usage.
> > 
> > 1. V4L2_CID_MTK_GET_BIN_INFO
> > - Provide the image output width & height in case
> > camera binning mode is enabled.
> 
> Could you explain with a bit more details what these binned width and height
> would mean? How would they relate to the video CAPTURE node width and height?
> Isn't this something that should be rather exposed as an appropriate
> selection rectangle, instead of custom controls?
> 

Frontal binning is Mediatek ISP P1 HW function and it is designed for
low power saving. The input sensor resolution will be divided by 2
including width & height in the source side of ISP HW and lower the
overall ISP throughput. So the ISP clock could be running in the low
speed for power saving. However, the image quality will be bad in this
mode. If the frontal binning is enabled by ISP driver, user space 3A
algorithm needs to know the new resolution of TG. Btw, current ISP
driver doesn't support this function. We just export this V4L2 contorl
for future support. But, after internal discussing, we will remove this
v4l2 control in next patch.

> > 
> > 2. V4L2_CID_MTK_RAW_PATH
> > - Export the path control of the main stream to user space.
> > One is pure raw and the other is processing raw.
> > The default value is 0 which outputs the pure raw bayer image
> > from sesnor, without image processing in ISP HW.
> 
> Is it just effectively a full processing bypass? The driver seems to only
> update the related configuration when the streaming starts. Can it be
> controlled per-frame?
> 
> Generally this sounds more like something that should be modelled using the
> media topology, similar to the example below.
> 
> /----------------\   /-------------------\   /--------------\
> |                |---|                   |   |              |
> | Capture Subdev |   | Processing Subdev |-o-| CAPTURE node |
> |                |-\ |                   | | |              |
> \----------------/ | \-------------------/ | \--------------/
>                    |                       |
>                    \-----------------------/
> 
> Then the userspace can select whether it wants the image from the capture
> interface directly or procesed by the ISP by configuring the media links
> appropriately.
> 
> The current limitation of this model is that it can't be easily configured
> per-frame, as media configurations are not included in the requests yet.
> 
> [snip]
> 

For this V4L2 control, it is designed for per stream, not per frame and
just implemented for future usage. As the same conclusion with the above
one, we will remove this, either in current version. If we have strong
requirement on this, we could adopt your suggestion to use media
topology to per stream control.


> > +static int handle_ctrl_get_bin_info(struct v4l2_ctrl *ctrl, int is_width)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct v4l2_format *fmt;
> > +
> > +	fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get bin info w*h:%d*%d is_width:%d",
> > +		fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, is_width);
> > +
> > +	if (is_width)
> > +		ctrl->val = fmt->fmt.pix_mp.width;
> > +	else
> > +		ctrl->val = fmt->fmt.pix_mp.height;
> 
> This seems to contradict to what the comment in the header says, because it just
> always returns the video node format and doesn't seem to care about whether
> binning is enabled or not.
> 

This is because binning mode is not supported in current driver's
implementation and no request on this function. We just create this for
future usage. In order to avoid confusion, we will remove this.

> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_get_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	ctrl->val = (p1_dev->isp_ctx.isp_raw_path == ISP_PROCESS_RAW_PATH);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "Get process raw:%d", ctrl->val);
> > +
> > +	return 0;
> > +}
> > +
> > +static int handle_ctrl_set_process_raw(struct v4l2_ctrl *ctrl)
> > +{
> > +	struct mtk_cam_dev *cam_dev = ctrl->priv;
> > +	struct isp_p1_device *p1_dev = get_p1_device(&cam_dev->pdev->dev);
> > +
> > +	p1_dev->isp_ctx.isp_raw_path = (ctrl->val) ?
> > +		ISP_PROCESS_RAW_PATH : ISP_PURE_RAW_PATH;
> > +	dev_dbg(&cam_dev->pdev->dev, "Set process raw:%d", ctrl->val);
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_g_ctrl(struct v4l2_ctrl *ctrl)
> 
> This is g_volatile_ctrl not, g_ctrl.
> 

Ok, thanks for your correction.

> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		handle_ctrl_get_process_raw(ctrl);
> > +		break;
> 
> No need to provide getters for non-volatile controls. The
> framework manages them.
> 

Ok, thanks for your correction.

> > +	case V4L2_CID_MTK_GET_BIN_WIDTH:
> > +		handle_ctrl_get_bin_info(ctrl, 1);
> > +		break;
> > +	case V4L2_CID_MTK_GET_BIN_HEIGTH:
> > +		handle_ctrl_get_bin_info(ctrl, 0);
> 
> It's trivial to get the value, so there isn't much benefit in having a
> function to do so, especially if one needs something like a is_width
> argument that further complicates the code.
> 

Ok, we will pay attention in this of implementation next time.

> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_MTK_PROCESSING_RAW:
> > +		return handle_ctrl_set_process_raw(ctrl);
> 
> Same as above. The operation is too trivial to deserve a function.
> 

Ok, got it.

> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_cam_dev_ctrl_ops = {
> > +	.g_volatile_ctrl = mtk_cam_dev_g_ctrl,
> > +	.s_ctrl = mtk_cam_dev_s_ctrl,
> > +};
> > +
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_PROCESSING_RAW,
> > +	.name = "MTK CAM PROCESSING RAW",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_WIDTH,
> > +	.name = "MTK CAM GET BIN WIDTH",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_WIDTH,
> > +	.max = IMG_MAX_WIDTH,
> > +	.step = 1,
> > +	.def = IMG_MAX_WIDTH,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_MTK_GET_BIN_HEIGTH,
> > +	.name = "MTK CAM GET BIN HEIGHT",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = IMG_MIN_HEIGHT,
> > +	.max = IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > +	},
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> 
> This should be checked at the end, after all the controls are added.
> 

Ok, got it.

> Best regards,
> Tomasz
> 

Based on above comments, we will remove mtk_cam_ctrl.h & mtk_cam_ctrl.c
in next patch to avoid over design.

Thanks for your comments.

Jungo



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-01  7:25       ` Tomasz Figa
  (?)
@ 2019-07-05  3:33         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  3:33 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, mchehab, rynn.wu, srv_heupstream, robh,
	ryan.yu, frankie.chiu, hverkuil, matthias.bgg, sj.huang,
	linux-mediatek, laurent.pinchart, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Tomasz,

On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > The purpose of this child device is to provide shared
> > memory management for exchanging tuning data between co-processor
> > and the Pass 1 unit of the camera ISP system, including cache
> > buffer handling.
> > 
> 
> Looks like we haven't really progressed on getting this replaced with
> something that doesn't require so much custom code. Let me propose something
> better then.
> 
> We already have a reserved memory mode in DT. If it has a compatible string
> of "shared-dma-pool", it would be registered in the coherent DMA framework
> [1]. That would make it available for consumer devices to look-up.
> 
> Now if we add a "memory-region" property to the SCP device node and point it
> to our reserved memory node, the SCP driver could look it up and hook to the
> DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> 
> That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> struct device use the coherent DMA ops, which operate on the assigned memory
> pool. With that, the P1 driver could just directly use those calls to
> manage the memory, without any custom code.
> 
> There is an example how this setup works in the s5p-mfc driver[3], but it
> needs to be noted that it creates child nodes, because it can have more than
> 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> need child nodes and could just use the SCP device directly.
> 
> [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> 
> Let me also post some specific comments below, in case we end up still
> needing any of the code.
> 

Thanks your suggestions.

After applying your suggestion in SCP device driver, we could remove
mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
to get SCP address. We could touch the buffer with this SCP address in
SCP processor. 

After that, we use dma_map_page_attrs with P1 device which supports
IOMMU domain to get IOVA address. For this address, we will assign
it to our ISP HW device to proceed.

Below is the snippet for ISP P1 compose buffer initialization.

	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_pa = addr;
	dev_dbg(dev, "scp addr:%pad\n", &addr);

	/* get iova address */
	addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
				  MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
				  DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		isp_ctx->scp_mem_pa = 0;
		dev_err(dev, "Failed to map scp iova\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_iova = addr;

Moreover, we have another meta input buffer usage.
For this kind of buffer, it will be allocated by V4L2 framework
with dma_alloc_coherent with SCP device. In order to get IOVA,
we will add dma_map_page_attrs in vb2_ops' buf_init function.
In buf_cleanup function, we will call dma_unmap_page_attrs function.

Based on these current implementation, do you think it is correct?
If we got any wrong, please let us know. 

Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
Maybe we could not get the correct sg_table.
Do you think it is a bug and need to fix?

For this new implementation, it will apply ISP P1 & P2 drivers[1].

[1] https://patchwork.kernel.org/cover/10905221/

> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
> >  3 files changed, 323 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > index 95f0b1c8fa1c..d545ca6f09c5 100644
> > --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
> >  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
> >  mtk-cam-isp-objs += mtk_cam.o
> >  mtk-cam-isp-objs += mtk_cam-scp.o
> > +mtk-cam-isp-objs += mtk_cam-smem.o
> >  
> >  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > new file mode 100644
> > index 000000000000..a9845668ce10
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > @@ -0,0 +1,304 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <asm/cacheflush.h>
> > +#include <linux/device.h>
> > +#include <linux/io.h>
> > +#include <linux/iommu.h>
> > +#include <linux/of.h>
> > +#include <linux/of_fdt.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam-smem.h"
> > +
> > +static struct dma_map_ops smem_dma_ops;
> > +
> > +struct mtk_cam_smem_dev {
> > +	struct device *dev;
> > +	struct sg_table sgt;
> > +	struct page **smem_pages;
> > +	dma_addr_t smem_base;
> > +	dma_addr_t smem_dma_base;
> > +	int smem_size;
> > +};
> > +
> > +struct dma_coherent_mem {
> > +	void		*virt_base;
> > +	dma_addr_t	device_base;
> > +	unsigned long	pfn_base;
> > +	int		size;
> > +	int		flags;
> > +	unsigned long	*bitmap;
> > +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> > +	bool		use_dev_dma_pfn_offset;
> > +};
> > +
> > +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> > +					 dma_addr_t iova)
> > +{
> > +	struct iommu_domain *domain;
> > +	dma_addr_t addr, limit;
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +
> > +	domain = iommu_get_domain_for_dev(dev);
> > +	if (!domain) {
> > +		dev_warn(dev, "No iommu group domain\n");
> > +		return 0;
> > +	}
> > +
> > +	addr = iommu_iova_to_phys(domain, iova);
> > +	limit = smem_dev->smem_base + smem_dev->smem_size;
> > +	if (addr < smem_dev->smem_base || addr >= limit) {
> > +		dev_err(dev,
> > +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> > +			&addr, &smem_dev->smem_base, &limit);
> > +		return 0;
> > +	}
> > +	return addr;
> > +}
> 
> This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
> and then the address wouldn't be valid, because it would point outside of
> the address range allowed for SCP to access and also it would only point to
> the first page backing the IOVA.
> 
> The correct approach would be to always carry SCP DMA address and IOVA
> together in some kind of struct describing such buffers.
> 

We will remove this function in next patch & handle this in buf_init
function.

> > +
> > +static int mtk_cam_smem_get_sgtable(struct device *dev,
> > +				    struct sg_table *sgt,
> > +				    void *cpu_addr, dma_addr_t dma_addr,
> > +				    size_t size, unsigned long attrs)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> > +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> > +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> > +
> > +	dev_dbg(dev,
> > +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> > +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> > +
> > +	return sg_alloc_table_from_pages(sgt,
> > +		smem_dev->smem_pages + pages_start,
> > +		pages_count, 0, size, GFP_KERNEL);
> > +}
> 
> This should be just dma_get_sgtable_attrs(), in the approach I suggested at
> the top.
> 

Yes, we will remove this in next patch.

> > +
> > +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> > +				       dma_addr_t addr)
> > +{
> > +	struct device *dev = smem_dev->dev;
> > +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> > +
> > +	if (addr < smem_dev->smem_base ||
> > +	    addr > smem_dev->smem_base + smem_dev->smem_size) {
> 
> This is off by one, should be >=.
> 
> Also, this wouldn't really guarantee the CPU access the caller is going to
> do is valid, because it doesn't consider the access operation size.
> 
> Generally I'd suggest designing the code so that it doesn't have to convert
> offset addresses between different address spaces.
> 

Yes, we will remove this in next patch.

> > +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> > +		return NULL;
> > +	}
> > +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> > +					 struct scatterlist *sgl, int nelems,
> > +					 enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_unmap_area(cpu_addr, sgl->length, dir);
> 
> It's not allowed to use this function anywhere outside of the DMA API
> internals. See the comment [4].
> 
> [4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> > +					    struct scatterlist *sgl,
> > +					    int nelems,
> > +					    enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_map_area(cpu_addr, sgl->length, dir);
> 
> Ditto.
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> > +				       struct dma_map_ops *smem_ops)
> > +{
> > +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> > +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> > +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> > +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> > +	set_dma_ops(dev, smem_ops);
> > +}
> > +
> > +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> > +{
> > +	u32 size_align, n_pages;
> > +	struct device *dev = smem_dev->dev;
> > +	struct sg_table *sgt = &smem_dev->sgt;
> > +	struct page **pages;
> > +	dma_addr_t dma_addr;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> > +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> > +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> > +		 __func__,
> > +		 smem_dev->dev,
> > +		 &smem_dev->smem_base,
> > +		 (smem_dev->smem_size / SZ_1M));
> > +
> > +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> > +	n_pages = size_align >> PAGE_SHIFT;
> > +
> > +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> > +	if (!pages)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < n_pages; i++)
> > +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> > +
> > +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> > +					size_align, GFP_KERNEL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> > +		goto fail_table_alloc;
> > +	}
> > +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> > +				      DMA_BIDIRECTIONAL,
> > +				      DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (!sgt->nents) {
> > +		dev_err(dev, "failed to dma sg map\n");
> > +		goto fail_map;
> > +	}
> > +
> > +	dma_addr = sg_dma_address(sgt->sgl);
> > +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> > +					  dma_addr, size_align,
> > +					  DMA_MEMORY_EXCLUSIVE);
> > +	if (ret) {
> > +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> > +		goto fail_map;
> > +	}
> > +
> > +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> > +		 &smem_dev->smem_base, &dma_addr, size_align);
> > +
> > +	smem_dev->smem_size = size_align;
> > +	smem_dev->smem_pages = pages;
> > +	smem_dev->smem_dma_base = dma_addr;
> > +
> > +	return 0;
> > +
> > +fail_map:
> > +	sg_free_table(sgt);
> > +fail_table_alloc:
> > +	while (n_pages--)
> > +		__free_page(pages[n_pages]);
> > +	kfree(pages);
> > +
> > +	return -ENOMEM;
> > +}
> > +
> > +/* DMA memory related helper functions */
> > +static void mtk_cam_memdev_release(struct device *dev)
> > +{
> > +	vb2_dma_contig_clear_max_seg_size(dev);
> > +}
> > +
> > +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> > +					     const char *name)
> > +{
> > +	struct device *child;
> > +	int ret;
> > +
> > +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> > +	if (!child)
> > +		return NULL;
> > +
> > +	child->parent = dev;
> > +	child->iommu_group = dev->iommu_group;
> 
> This isn't something that can be set explicitly. It's an internal field of
> the IOMMU subsystem.
> 
> > +	child->release = mtk_cam_memdev_release;
> > +	dev_set_name(child, name);
> > +	set_dma_ops(child, get_dma_ops(dev));
> > +	child->dma_mask = dev->dma_mask;
> > +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> > +	if (ret)
> > +		return NULL;
> > +
> > +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> > +
> > +	if (device_register(child)) {
> > +		device_del(child);
> > +		return NULL;
> > +	}
> > +
> > +	return child;
> > +}
> 
> We shouldn't need child devices, just one SCP device, as I mentioned above.
> 

Ok, got your point. Just keep one single SCP device for single reserved
memory range.

> > +
> > +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	u32 size;
> > +	dma_addr_t addr;
> > +
> > +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> > +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> > +	if (!isp_ctx->scp_mem_pa || !size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
> 
> This isn't something that deserves the "info" log level. Should be "dbg"
> or removed.
> 

Ok, we will change the log level from info to debug.

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

Thanks for your valued comments.

Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  3:33         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  3:33 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Tomasz,

On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > The purpose of this child device is to provide shared
> > memory management for exchanging tuning data between co-processor
> > and the Pass 1 unit of the camera ISP system, including cache
> > buffer handling.
> > 
> 
> Looks like we haven't really progressed on getting this replaced with
> something that doesn't require so much custom code. Let me propose something
> better then.
> 
> We already have a reserved memory mode in DT. If it has a compatible string
> of "shared-dma-pool", it would be registered in the coherent DMA framework
> [1]. That would make it available for consumer devices to look-up.
> 
> Now if we add a "memory-region" property to the SCP device node and point it
> to our reserved memory node, the SCP driver could look it up and hook to the
> DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> 
> That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> struct device use the coherent DMA ops, which operate on the assigned memory
> pool. With that, the P1 driver could just directly use those calls to
> manage the memory, without any custom code.
> 
> There is an example how this setup works in the s5p-mfc driver[3], but it
> needs to be noted that it creates child nodes, because it can have more than
> 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> need child nodes and could just use the SCP device directly.
> 
> [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> 
> Let me also post some specific comments below, in case we end up still
> needing any of the code.
> 

Thanks your suggestions.

After applying your suggestion in SCP device driver, we could remove
mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
to get SCP address. We could touch the buffer with this SCP address in
SCP processor. 

After that, we use dma_map_page_attrs with P1 device which supports
IOMMU domain to get IOVA address. For this address, we will assign
it to our ISP HW device to proceed.

Below is the snippet for ISP P1 compose buffer initialization.

	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_pa = addr;
	dev_dbg(dev, "scp addr:%pad\n", &addr);

	/* get iova address */
	addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
				  MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
				  DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		isp_ctx->scp_mem_pa = 0;
		dev_err(dev, "Failed to map scp iova\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_iova = addr;

Moreover, we have another meta input buffer usage.
For this kind of buffer, it will be allocated by V4L2 framework
with dma_alloc_coherent with SCP device. In order to get IOVA,
we will add dma_map_page_attrs in vb2_ops' buf_init function.
In buf_cleanup function, we will call dma_unmap_page_attrs function.

Based on these current implementation, do you think it is correct?
If we got any wrong, please let us know. 

Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
Maybe we could not get the correct sg_table.
Do you think it is a bug and need to fix?

For this new implementation, it will apply ISP P1 & P2 drivers[1].

[1] https://patchwork.kernel.org/cover/10905221/

> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
> >  3 files changed, 323 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > index 95f0b1c8fa1c..d545ca6f09c5 100644
> > --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
> >  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
> >  mtk-cam-isp-objs += mtk_cam.o
> >  mtk-cam-isp-objs += mtk_cam-scp.o
> > +mtk-cam-isp-objs += mtk_cam-smem.o
> >  
> >  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > new file mode 100644
> > index 000000000000..a9845668ce10
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > @@ -0,0 +1,304 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <asm/cacheflush.h>
> > +#include <linux/device.h>
> > +#include <linux/io.h>
> > +#include <linux/iommu.h>
> > +#include <linux/of.h>
> > +#include <linux/of_fdt.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam-smem.h"
> > +
> > +static struct dma_map_ops smem_dma_ops;
> > +
> > +struct mtk_cam_smem_dev {
> > +	struct device *dev;
> > +	struct sg_table sgt;
> > +	struct page **smem_pages;
> > +	dma_addr_t smem_base;
> > +	dma_addr_t smem_dma_base;
> > +	int smem_size;
> > +};
> > +
> > +struct dma_coherent_mem {
> > +	void		*virt_base;
> > +	dma_addr_t	device_base;
> > +	unsigned long	pfn_base;
> > +	int		size;
> > +	int		flags;
> > +	unsigned long	*bitmap;
> > +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> > +	bool		use_dev_dma_pfn_offset;
> > +};
> > +
> > +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> > +					 dma_addr_t iova)
> > +{
> > +	struct iommu_domain *domain;
> > +	dma_addr_t addr, limit;
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +
> > +	domain = iommu_get_domain_for_dev(dev);
> > +	if (!domain) {
> > +		dev_warn(dev, "No iommu group domain\n");
> > +		return 0;
> > +	}
> > +
> > +	addr = iommu_iova_to_phys(domain, iova);
> > +	limit = smem_dev->smem_base + smem_dev->smem_size;
> > +	if (addr < smem_dev->smem_base || addr >= limit) {
> > +		dev_err(dev,
> > +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> > +			&addr, &smem_dev->smem_base, &limit);
> > +		return 0;
> > +	}
> > +	return addr;
> > +}
> 
> This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
> and then the address wouldn't be valid, because it would point outside of
> the address range allowed for SCP to access and also it would only point to
> the first page backing the IOVA.
> 
> The correct approach would be to always carry SCP DMA address and IOVA
> together in some kind of struct describing such buffers.
> 

We will remove this function in next patch & handle this in buf_init
function.

> > +
> > +static int mtk_cam_smem_get_sgtable(struct device *dev,
> > +				    struct sg_table *sgt,
> > +				    void *cpu_addr, dma_addr_t dma_addr,
> > +				    size_t size, unsigned long attrs)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> > +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> > +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> > +
> > +	dev_dbg(dev,
> > +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> > +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> > +
> > +	return sg_alloc_table_from_pages(sgt,
> > +		smem_dev->smem_pages + pages_start,
> > +		pages_count, 0, size, GFP_KERNEL);
> > +}
> 
> This should be just dma_get_sgtable_attrs(), in the approach I suggested at
> the top.
> 

Yes, we will remove this in next patch.

> > +
> > +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> > +				       dma_addr_t addr)
> > +{
> > +	struct device *dev = smem_dev->dev;
> > +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> > +
> > +	if (addr < smem_dev->smem_base ||
> > +	    addr > smem_dev->smem_base + smem_dev->smem_size) {
> 
> This is off by one, should be >=.
> 
> Also, this wouldn't really guarantee the CPU access the caller is going to
> do is valid, because it doesn't consider the access operation size.
> 
> Generally I'd suggest designing the code so that it doesn't have to convert
> offset addresses between different address spaces.
> 

Yes, we will remove this in next patch.

> > +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> > +		return NULL;
> > +	}
> > +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> > +					 struct scatterlist *sgl, int nelems,
> > +					 enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_unmap_area(cpu_addr, sgl->length, dir);
> 
> It's not allowed to use this function anywhere outside of the DMA API
> internals. See the comment [4].
> 
> [4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> > +					    struct scatterlist *sgl,
> > +					    int nelems,
> > +					    enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_map_area(cpu_addr, sgl->length, dir);
> 
> Ditto.
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> > +				       struct dma_map_ops *smem_ops)
> > +{
> > +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> > +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> > +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> > +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> > +	set_dma_ops(dev, smem_ops);
> > +}
> > +
> > +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> > +{
> > +	u32 size_align, n_pages;
> > +	struct device *dev = smem_dev->dev;
> > +	struct sg_table *sgt = &smem_dev->sgt;
> > +	struct page **pages;
> > +	dma_addr_t dma_addr;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> > +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> > +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> > +		 __func__,
> > +		 smem_dev->dev,
> > +		 &smem_dev->smem_base,
> > +		 (smem_dev->smem_size / SZ_1M));
> > +
> > +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> > +	n_pages = size_align >> PAGE_SHIFT;
> > +
> > +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> > +	if (!pages)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < n_pages; i++)
> > +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> > +
> > +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> > +					size_align, GFP_KERNEL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> > +		goto fail_table_alloc;
> > +	}
> > +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> > +				      DMA_BIDIRECTIONAL,
> > +				      DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (!sgt->nents) {
> > +		dev_err(dev, "failed to dma sg map\n");
> > +		goto fail_map;
> > +	}
> > +
> > +	dma_addr = sg_dma_address(sgt->sgl);
> > +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> > +					  dma_addr, size_align,
> > +					  DMA_MEMORY_EXCLUSIVE);
> > +	if (ret) {
> > +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> > +		goto fail_map;
> > +	}
> > +
> > +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> > +		 &smem_dev->smem_base, &dma_addr, size_align);
> > +
> > +	smem_dev->smem_size = size_align;
> > +	smem_dev->smem_pages = pages;
> > +	smem_dev->smem_dma_base = dma_addr;
> > +
> > +	return 0;
> > +
> > +fail_map:
> > +	sg_free_table(sgt);
> > +fail_table_alloc:
> > +	while (n_pages--)
> > +		__free_page(pages[n_pages]);
> > +	kfree(pages);
> > +
> > +	return -ENOMEM;
> > +}
> > +
> > +/* DMA memory related helper functions */
> > +static void mtk_cam_memdev_release(struct device *dev)
> > +{
> > +	vb2_dma_contig_clear_max_seg_size(dev);
> > +}
> > +
> > +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> > +					     const char *name)
> > +{
> > +	struct device *child;
> > +	int ret;
> > +
> > +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> > +	if (!child)
> > +		return NULL;
> > +
> > +	child->parent = dev;
> > +	child->iommu_group = dev->iommu_group;
> 
> This isn't something that can be set explicitly. It's an internal field of
> the IOMMU subsystem.
> 
> > +	child->release = mtk_cam_memdev_release;
> > +	dev_set_name(child, name);
> > +	set_dma_ops(child, get_dma_ops(dev));
> > +	child->dma_mask = dev->dma_mask;
> > +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> > +	if (ret)
> > +		return NULL;
> > +
> > +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> > +
> > +	if (device_register(child)) {
> > +		device_del(child);
> > +		return NULL;
> > +	}
> > +
> > +	return child;
> > +}
> 
> We shouldn't need child devices, just one SCP device, as I mentioned above.
> 

Ok, got your point. Just keep one single SCP device for single reserved
memory range.

> > +
> > +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	u32 size;
> > +	dma_addr_t addr;
> > +
> > +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> > +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> > +	if (!isp_ctx->scp_mem_pa || !size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
> 
> This isn't something that deserves the "info" log level. Should be "dbg"
> or removed.
> 

Ok, we will change the log level from info to debug.

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

Thanks for your valued comments.

Best regards,


Jungo



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  3:33         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  3:33 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, mchehab, rynn.wu, srv_heupstream, robh,
	ryan.yu, frankie.chiu, hverkuil, matthias.bgg, sj.huang,
	linux-mediatek, laurent.pinchart, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Tomasz,

On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > The purpose of this child device is to provide shared
> > memory management for exchanging tuning data between co-processor
> > and the Pass 1 unit of the camera ISP system, including cache
> > buffer handling.
> > 
> 
> Looks like we haven't really progressed on getting this replaced with
> something that doesn't require so much custom code. Let me propose something
> better then.
> 
> We already have a reserved memory mode in DT. If it has a compatible string
> of "shared-dma-pool", it would be registered in the coherent DMA framework
> [1]. That would make it available for consumer devices to look-up.
> 
> Now if we add a "memory-region" property to the SCP device node and point it
> to our reserved memory node, the SCP driver could look it up and hook to the
> DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> 
> That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> struct device use the coherent DMA ops, which operate on the assigned memory
> pool. With that, the P1 driver could just directly use those calls to
> manage the memory, without any custom code.
> 
> There is an example how this setup works in the s5p-mfc driver[3], but it
> needs to be noted that it creates child nodes, because it can have more than
> 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> need child nodes and could just use the SCP device directly.
> 
> [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> 
> Let me also post some specific comments below, in case we end up still
> needing any of the code.
> 

Thanks your suggestions.

After applying your suggestion in SCP device driver, we could remove
mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
to get SCP address. We could touch the buffer with this SCP address in
SCP processor. 

After that, we use dma_map_page_attrs with P1 device which supports
IOMMU domain to get IOVA address. For this address, we will assign
it to our ISP HW device to proceed.

Below is the snippet for ISP P1 compose buffer initialization.

	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_pa = addr;
	dev_dbg(dev, "scp addr:%pad\n", &addr);

	/* get iova address */
	addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
				  MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
				  DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		isp_ctx->scp_mem_pa = 0;
		dev_err(dev, "Failed to map scp iova\n");
		return -ENOMEM;
	}
	isp_ctx->scp_mem_iova = addr;

Moreover, we have another meta input buffer usage.
For this kind of buffer, it will be allocated by V4L2 framework
with dma_alloc_coherent with SCP device. In order to get IOVA,
we will add dma_map_page_attrs in vb2_ops' buf_init function.
In buf_cleanup function, we will call dma_unmap_page_attrs function.

Based on these current implementation, do you think it is correct?
If we got any wrong, please let us know. 

Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
Maybe we could not get the correct sg_table.
Do you think it is a bug and need to fix?

For this new implementation, it will apply ISP P1 & P2 drivers[1].

[1] https://patchwork.kernel.org/cover/10905221/

> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.c         | 304 ++++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-smem.h         |  18 ++
> >  3 files changed, 323 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > index 95f0b1c8fa1c..d545ca6f09c5 100644
> > --- a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -4,5 +4,6 @@ mtk-cam-isp-objs += mtk_cam-ctrl.o
> >  mtk-cam-isp-objs += mtk_cam-v4l2-util.o
> >  mtk-cam-isp-objs += mtk_cam.o
> >  mtk-cam-isp-objs += mtk_cam-scp.o
> > +mtk-cam-isp-objs += mtk_cam-smem.o
> >  
> >  obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > new file mode 100644
> > index 000000000000..a9845668ce10
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-smem.c
> > @@ -0,0 +1,304 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <asm/cacheflush.h>
> > +#include <linux/device.h>
> > +#include <linux/io.h>
> > +#include <linux/iommu.h>
> > +#include <linux/of.h>
> > +#include <linux/of_fdt.h>
> > +#include <linux/of_reserved_mem.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam-smem.h"
> > +
> > +static struct dma_map_ops smem_dma_ops;
> > +
> > +struct mtk_cam_smem_dev {
> > +	struct device *dev;
> > +	struct sg_table sgt;
> > +	struct page **smem_pages;
> > +	dma_addr_t smem_base;
> > +	dma_addr_t smem_dma_base;
> > +	int smem_size;
> > +};
> > +
> > +struct dma_coherent_mem {
> > +	void		*virt_base;
> > +	dma_addr_t	device_base;
> > +	unsigned long	pfn_base;
> > +	int		size;
> > +	int		flags;
> > +	unsigned long	*bitmap;
> > +	spinlock_t	spinlock; /* dma_coherent_mem attributes protection */
> > +	bool		use_dev_dma_pfn_offset;
> > +};
> > +
> > +dma_addr_t mtk_cam_smem_iova_to_scp_addr(struct device *dev,
> > +					 dma_addr_t iova)
> > +{
> > +	struct iommu_domain *domain;
> > +	dma_addr_t addr, limit;
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +
> > +	domain = iommu_get_domain_for_dev(dev);
> > +	if (!domain) {
> > +		dev_warn(dev, "No iommu group domain\n");
> > +		return 0;
> > +	}
> > +
> > +	addr = iommu_iova_to_phys(domain, iova);
> > +	limit = smem_dev->smem_base + smem_dev->smem_size;
> > +	if (addr < smem_dev->smem_base || addr >= limit) {
> > +		dev_err(dev,
> > +			"Unexpected scp_addr:%pad must >= %pad and < %pad)\n",
> > +			&addr, &smem_dev->smem_base, &limit);
> > +		return 0;
> > +	}
> > +	return addr;
> > +}
> 
> This isn't correct. One could pass an IOVA that wasn't allocated for the SCP
> and then the address wouldn't be valid, because it would point outside of
> the address range allowed for SCP to access and also it would only point to
> the first page backing the IOVA.
> 
> The correct approach would be to always carry SCP DMA address and IOVA
> together in some kind of struct describing such buffers.
> 

We will remove this function in next patch & handle this in buf_init
function.

> > +
> > +static int mtk_cam_smem_get_sgtable(struct device *dev,
> > +				    struct sg_table *sgt,
> > +				    void *cpu_addr, dma_addr_t dma_addr,
> > +				    size_t size, unsigned long attrs)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	size_t pages_count = PAGE_ALIGN(size) >> PAGE_SHIFT;
> > +	dma_addr_t scp_addr = mtk_cam_smem_iova_to_scp_addr(dev, dma_addr);
> > +	u32 pages_start = (scp_addr - smem_dev->smem_base) >> PAGE_SHIFT;
> > +
> > +	dev_dbg(dev,
> > +		"%s:page:%u va:%pK scp addr:%pad, aligned size:%zu pages:%zu\n",
> > +		__func__, pages_start, cpu_addr, &scp_addr, size, pages_count);
> > +
> > +	return sg_alloc_table_from_pages(sgt,
> > +		smem_dev->smem_pages + pages_start,
> > +		pages_count, 0, size, GFP_KERNEL);
> > +}
> 
> This should be just dma_get_sgtable_attrs(), in the approach I suggested at
> the top.
> 

Yes, we will remove this in next patch.

> > +
> > +static void *mtk_cam_smem_get_cpu_addr(struct mtk_cam_smem_dev *smem_dev,
> > +				       dma_addr_t addr)
> > +{
> > +	struct device *dev = smem_dev->dev;
> > +	struct dma_coherent_mem *dma_mem = dev->dma_mem;
> > +
> > +	if (addr < smem_dev->smem_base ||
> > +	    addr > smem_dev->smem_base + smem_dev->smem_size) {
> 
> This is off by one, should be >=.
> 
> Also, this wouldn't really guarantee the CPU access the caller is going to
> do is valid, because it doesn't consider the access operation size.
> 
> Generally I'd suggest designing the code so that it doesn't have to convert
> offset addresses between different address spaces.
> 

Yes, we will remove this in next patch.

> > +		dev_err(dev, "Invalid scp_addr %pad from sg\n", &addr);
> > +		return NULL;
> > +	}
> > +	return dma_mem->virt_base + (addr - smem_dev->smem_base);
> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_cpu(struct device *dev,
> > +					 struct scatterlist *sgl, int nelems,
> > +					 enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_unmap_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_unmap_area(cpu_addr, sgl->length, dir);
> 
> It's not allowed to use this function anywhere outside of the DMA API
> internals. See the comment [4].
> 
> [4] https://elixir.bootlin.com/linux/v5.2-rc7/source/arch/arm64/include/asm/cacheflush.h#L112
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_sync_sg_for_device(struct device *dev,
> > +					    struct scatterlist *sgl,
> > +					    int nelems,
> > +					    enum dma_data_direction dir)
> > +{
> > +	struct mtk_cam_smem_dev *smem_dev = dev_get_drvdata(dev);
> > +	dma_addr_t scp_addr = sg_phys(sgl);
> > +	void *cpu_addr = mtk_cam_smem_get_cpu_addr(smem_dev, scp_addr);
> > +
> > +	dev_dbg(dev,
> > +		"__dma_map_area:scp_addr:%pad,vaddr:%pK,size:%d,dir:%d\n",
> > +		&scp_addr, cpu_addr, sgl->length, dir);
> > +	__dma_map_area(cpu_addr, sgl->length, dir);
> 
> Ditto.
> 

Ok, got it and remove this next patch.

> > +}
> > +
> > +static void mtk_cam_smem_setup_dma_ops(struct device *dev,
> > +				       struct dma_map_ops *smem_ops)
> > +{
> > +	memcpy((void *)smem_ops, dev->dma_ops, sizeof(*smem_ops));
> > +	smem_ops->get_sgtable = mtk_cam_smem_get_sgtable;
> > +	smem_ops->sync_sg_for_device = mtk_cam_smem_sync_sg_for_device;
> > +	smem_ops->sync_sg_for_cpu = mtk_cam_smem_sync_sg_for_cpu;
> > +	set_dma_ops(dev, smem_ops);
> > +}
> > +
> > +static int mtk_cam_reserved_drm_sg_init(struct mtk_cam_smem_dev *smem_dev)
> > +{
> > +	u32 size_align, n_pages;
> > +	struct device *dev = smem_dev->dev;
> > +	struct sg_table *sgt = &smem_dev->sgt;
> > +	struct page **pages;
> > +	dma_addr_t dma_addr;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	smem_dev->smem_base = scp_get_reserve_mem_phys(SCP_ISP_MEM2_ID);
> > +	smem_dev->smem_size = scp_get_reserve_mem_size(SCP_ISP_MEM2_ID);
> > +	if (!smem_dev->smem_base || !smem_dev->smem_size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "%s dev:0x%pK base:%pad size:%u MiB\n",
> > +		 __func__,
> > +		 smem_dev->dev,
> > +		 &smem_dev->smem_base,
> > +		 (smem_dev->smem_size / SZ_1M));
> > +
> > +	size_align = PAGE_ALIGN(smem_dev->smem_size);
> > +	n_pages = size_align >> PAGE_SHIFT;
> > +
> > +	pages = kmalloc_array(n_pages, sizeof(struct page *), GFP_KERNEL);
> > +	if (!pages)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < n_pages; i++)
> > +		pages[i] = phys_to_page(smem_dev->smem_base + i * PAGE_SIZE);
> > +
> > +	ret = sg_alloc_table_from_pages(sgt, pages, n_pages, 0,
> > +					size_align, GFP_KERNEL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to alloca sg table:%d\n", ret);
> > +		goto fail_table_alloc;
> > +	}
> > +	sgt->nents = dma_map_sg_attrs(dev, sgt->sgl, sgt->orig_nents,
> > +				      DMA_BIDIRECTIONAL,
> > +				      DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (!sgt->nents) {
> > +		dev_err(dev, "failed to dma sg map\n");
> > +		goto fail_map;
> > +	}
> > +
> > +	dma_addr = sg_dma_address(sgt->sgl);
> > +	ret = dma_declare_coherent_memory(dev, smem_dev->smem_base,
> > +					  dma_addr, size_align,
> > +					  DMA_MEMORY_EXCLUSIVE);
> > +	if (ret) {
> > +		dev_err(dev, "Unable to declare smem  memory:%d\n", ret);
> > +		goto fail_map;
> > +	}
> > +
> > +	dev_info(dev, "Coherent mem pa:%pad/%pad, size:%d\n",
> > +		 &smem_dev->smem_base, &dma_addr, size_align);
> > +
> > +	smem_dev->smem_size = size_align;
> > +	smem_dev->smem_pages = pages;
> > +	smem_dev->smem_dma_base = dma_addr;
> > +
> > +	return 0;
> > +
> > +fail_map:
> > +	sg_free_table(sgt);
> > +fail_table_alloc:
> > +	while (n_pages--)
> > +		__free_page(pages[n_pages]);
> > +	kfree(pages);
> > +
> > +	return -ENOMEM;
> > +}
> > +
> > +/* DMA memory related helper functions */
> > +static void mtk_cam_memdev_release(struct device *dev)
> > +{
> > +	vb2_dma_contig_clear_max_seg_size(dev);
> > +}
> > +
> > +static struct device *mtk_cam_alloc_smem_dev(struct device *dev,
> > +					     const char *name)
> > +{
> > +	struct device *child;
> > +	int ret;
> > +
> > +	child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
> > +	if (!child)
> > +		return NULL;
> > +
> > +	child->parent = dev;
> > +	child->iommu_group = dev->iommu_group;
> 
> This isn't something that can be set explicitly. It's an internal field of
> the IOMMU subsystem.
> 
> > +	child->release = mtk_cam_memdev_release;
> > +	dev_set_name(child, name);
> > +	set_dma_ops(child, get_dma_ops(dev));
> > +	child->dma_mask = dev->dma_mask;
> > +	ret = dma_set_coherent_mask(child, DMA_BIT_MASK(32));
> > +	if (ret)
> > +		return NULL;
> > +
> > +	vb2_dma_contig_set_max_seg_size(child, DMA_BIT_MASK(32));
> > +
> > +	if (device_register(child)) {
> > +		device_del(child);
> > +		return NULL;
> > +	}
> > +
> > +	return child;
> > +}
> 
> We shouldn't need child devices, just one SCP device, as I mentioned above.
> 

Ok, got your point. Just keep one single SCP device for single reserved
memory range.

> > +
> > +static int mtk_cam_composer_dma_init(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	u32 size;
> > +	dma_addr_t addr;
> > +
> > +	isp_ctx->scp_mem_pa = scp_get_reserve_mem_phys(SCP_ISP_MEM_ID);
> > +	size = PAGE_ALIGN(scp_get_reserve_mem_size(SCP_ISP_MEM_ID));
> > +	if (!isp_ctx->scp_mem_pa || !size)
> > +		return -EPROBE_DEFER;
> > +
> > +	dev_info(dev, "scp addr:%pad size:0x%x\n", &isp_ctx->scp_mem_pa, size);
> 
> This isn't something that deserves the "info" log level. Should be "dbg"
> or removed.
> 

Ok, we will change the log level from info to debug.

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

Thanks for your valued comments.

Best regards,


Jungo



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-05  3:33         ` Jungo Lin
  (?)
@ 2019-07-05  4:22           ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-05  4:22 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元)

Hi Jungo,

On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Tomasz,
>
> On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > The purpose of this child device is to provide shared
> > > memory management for exchanging tuning data between co-processor
> > > and the Pass 1 unit of the camera ISP system, including cache
> > > buffer handling.
> > >
> >
> > Looks like we haven't really progressed on getting this replaced with
> > something that doesn't require so much custom code. Let me propose something
> > better then.
> >
> > We already have a reserved memory mode in DT. If it has a compatible string
> > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > [1]. That would make it available for consumer devices to look-up.
> >
> > Now if we add a "memory-region" property to the SCP device node and point it
> > to our reserved memory node, the SCP driver could look it up and hook to the
> > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> >
> > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > struct device use the coherent DMA ops, which operate on the assigned memory
> > pool. With that, the P1 driver could just directly use those calls to
> > manage the memory, without any custom code.
> >
> > There is an example how this setup works in the s5p-mfc driver[3], but it
> > needs to be noted that it creates child nodes, because it can have more than
> > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > need child nodes and could just use the SCP device directly.
> >
> > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> >
> > Let me also post some specific comments below, in case we end up still
> > needing any of the code.
> >
>
> Thanks your suggestions.
>
> After applying your suggestion in SCP device driver, we could remove
> mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> to get SCP address. We could touch the buffer with this SCP address in
> SCP processor.
>
> After that, we use dma_map_page_attrs with P1 device which supports
> IOMMU domain to get IOVA address. For this address, we will assign
> it to our ISP HW device to proceed.
>
> Below is the snippet for ISP P1 compose buffer initialization.
>
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_pa = addr;

addr contains a DMA address, not a physical address. Could we call it
scp_mem_dma instead?

>         dev_dbg(dev, "scp addr:%pad\n", &addr);
>
>         /* get iova address */
>         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,

addr is a DMA address, so phys_to_page() can't be called on it. The
simplest thing here would be to use dma_map_single() with ptr as the
CPU address expected.

>                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
>                                   DMA_ATTR_SKIP_CPU_SYNC);
>         if (dma_mapping_error(dev, addr)) {
>                 isp_ctx->scp_mem_pa = 0;

We also need to free the allocated memory.

>                 dev_err(dev, "Failed to map scp iova\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_iova = addr;
>
> Moreover, we have another meta input buffer usage.
> For this kind of buffer, it will be allocated by V4L2 framework
> with dma_alloc_coherent with SCP device. In order to get IOVA,
> we will add dma_map_page_attrs in vb2_ops' buf_init function.
> In buf_cleanup function, we will call dma_unmap_page_attrs function.

As per above, we don't have access to the struct page we want to map.
We probably want to get the CPU VA using vb2_plane_vaddr() and call
dma_map_single() instead.

>
> Based on these current implementation, do you think it is correct?
> If we got any wrong, please let us know.
>
> Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> Maybe we could not get the correct sg_table.
> Do you think it is a bug and need to fix?

I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
that don't need to be accessed from the kernel anyway, to avoid
unnecessary kernel mapping operations. However, for coherent memory
pool, it doesn't change anything, because the memory always has a
kernel mapping. We also need the kernel virtual address for
dma_map_single(). Also the flag doesn't eliminate the need to do the
sync, e.g. if the userspace accesses the buffer.

Could you give me more information about the failure you're seeing?
Where is the dma_sync_sg_for_device() called from? Where do you get
the sgtable from?

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  4:22           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-05  4:22 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi Jungo,

On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Tomasz,
>
> On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > The purpose of this child device is to provide shared
> > > memory management for exchanging tuning data between co-processor
> > > and the Pass 1 unit of the camera ISP system, including cache
> > > buffer handling.
> > >
> >
> > Looks like we haven't really progressed on getting this replaced with
> > something that doesn't require so much custom code. Let me propose something
> > better then.
> >
> > We already have a reserved memory mode in DT. If it has a compatible string
> > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > [1]. That would make it available for consumer devices to look-up.
> >
> > Now if we add a "memory-region" property to the SCP device node and point it
> > to our reserved memory node, the SCP driver could look it up and hook to the
> > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> >
> > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > struct device use the coherent DMA ops, which operate on the assigned memory
> > pool. With that, the P1 driver could just directly use those calls to
> > manage the memory, without any custom code.
> >
> > There is an example how this setup works in the s5p-mfc driver[3], but it
> > needs to be noted that it creates child nodes, because it can have more than
> > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > need child nodes and could just use the SCP device directly.
> >
> > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> >
> > Let me also post some specific comments below, in case we end up still
> > needing any of the code.
> >
>
> Thanks your suggestions.
>
> After applying your suggestion in SCP device driver, we could remove
> mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> to get SCP address. We could touch the buffer with this SCP address in
> SCP processor.
>
> After that, we use dma_map_page_attrs with P1 device which supports
> IOMMU domain to get IOVA address. For this address, we will assign
> it to our ISP HW device to proceed.
>
> Below is the snippet for ISP P1 compose buffer initialization.
>
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_pa = addr;

addr contains a DMA address, not a physical address. Could we call it
scp_mem_dma instead?

>         dev_dbg(dev, "scp addr:%pad\n", &addr);
>
>         /* get iova address */
>         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,

addr is a DMA address, so phys_to_page() can't be called on it. The
simplest thing here would be to use dma_map_single() with ptr as the
CPU address expected.

>                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
>                                   DMA_ATTR_SKIP_CPU_SYNC);
>         if (dma_mapping_error(dev, addr)) {
>                 isp_ctx->scp_mem_pa = 0;

We also need to free the allocated memory.

>                 dev_err(dev, "Failed to map scp iova\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_iova = addr;
>
> Moreover, we have another meta input buffer usage.
> For this kind of buffer, it will be allocated by V4L2 framework
> with dma_alloc_coherent with SCP device. In order to get IOVA,
> we will add dma_map_page_attrs in vb2_ops' buf_init function.
> In buf_cleanup function, we will call dma_unmap_page_attrs function.

As per above, we don't have access to the struct page we want to map.
We probably want to get the CPU VA using vb2_plane_vaddr() and call
dma_map_single() instead.

>
> Based on these current implementation, do you think it is correct?
> If we got any wrong, please let us know.
>
> Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> Maybe we could not get the correct sg_table.
> Do you think it is a bug and need to fix?

I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
that don't need to be accessed from the kernel anyway, to avoid
unnecessary kernel mapping operations. However, for coherent memory
pool, it doesn't change anything, because the memory always has a
kernel mapping. We also need the kernel virtual address for
dma_map_single(). Also the flag doesn't eliminate the need to do the
sync, e.g. if the userspace accesses the buffer.

Could you give me more information about the failure you're seeing?
Where is the dma_sync_sg_for_device() called from? Where do you get
the sgtable from?

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  4:22           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-05  4:22 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Tomasz,
>
> On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > The purpose of this child device is to provide shared
> > > memory management for exchanging tuning data between co-processor
> > > and the Pass 1 unit of the camera ISP system, including cache
> > > buffer handling.
> > >
> >
> > Looks like we haven't really progressed on getting this replaced with
> > something that doesn't require so much custom code. Let me propose something
> > better then.
> >
> > We already have a reserved memory mode in DT. If it has a compatible string
> > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > [1]. That would make it available for consumer devices to look-up.
> >
> > Now if we add a "memory-region" property to the SCP device node and point it
> > to our reserved memory node, the SCP driver could look it up and hook to the
> > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> >
> > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > struct device use the coherent DMA ops, which operate on the assigned memory
> > pool. With that, the P1 driver could just directly use those calls to
> > manage the memory, without any custom code.
> >
> > There is an example how this setup works in the s5p-mfc driver[3], but it
> > needs to be noted that it creates child nodes, because it can have more than
> > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > need child nodes and could just use the SCP device directly.
> >
> > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> >
> > Let me also post some specific comments below, in case we end up still
> > needing any of the code.
> >
>
> Thanks your suggestions.
>
> After applying your suggestion in SCP device driver, we could remove
> mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> to get SCP address. We could touch the buffer with this SCP address in
> SCP processor.
>
> After that, we use dma_map_page_attrs with P1 device which supports
> IOMMU domain to get IOVA address. For this address, we will assign
> it to our ISP HW device to proceed.
>
> Below is the snippet for ISP P1 compose buffer initialization.
>
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_pa = addr;

addr contains a DMA address, not a physical address. Could we call it
scp_mem_dma instead?

>         dev_dbg(dev, "scp addr:%pad\n", &addr);
>
>         /* get iova address */
>         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,

addr is a DMA address, so phys_to_page() can't be called on it. The
simplest thing here would be to use dma_map_single() with ptr as the
CPU address expected.

>                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
>                                   DMA_ATTR_SKIP_CPU_SYNC);
>         if (dma_mapping_error(dev, addr)) {
>                 isp_ctx->scp_mem_pa = 0;

We also need to free the allocated memory.

>                 dev_err(dev, "Failed to map scp iova\n");
>                 return -ENOMEM;
>         }
>         isp_ctx->scp_mem_iova = addr;
>
> Moreover, we have another meta input buffer usage.
> For this kind of buffer, it will be allocated by V4L2 framework
> with dma_alloc_coherent with SCP device. In order to get IOVA,
> we will add dma_map_page_attrs in vb2_ops' buf_init function.
> In buf_cleanup function, we will call dma_unmap_page_attrs function.

As per above, we don't have access to the struct page we want to map.
We probably want to get the CPU VA using vb2_plane_vaddr() and call
dma_map_single() instead.

>
> Based on these current implementation, do you think it is correct?
> If we got any wrong, please let us know.
>
> Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> Maybe we could not get the correct sg_table.
> Do you think it is a bug and need to fix?

I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
that don't need to be accessed from the kernel anyway, to avoid
unnecessary kernel mapping operations. However, for coherent memory
pool, it doesn't change anything, because the memory always has a
kernel mapping. We also need the kernel virtual address for
dma_map_single(). Also the flag doesn't eliminate the need to do the
sync, e.g. if the userspace accesses the buffer.

Could you give me more information about the failure you're seeing?
Where is the dma_sync_sg_for_device() called from? Where do you get
the sgtable from?

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-05  4:22           ` [RFC,v3 " Tomasz Figa
  (?)
@ 2019-07-05  5:44               ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  5:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Frederic Chen (陳俊元),
	list-Y9sIeH5OGRo

Hi, Tomasz:
On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> >
> > Hi Tomasz,
> >
> > On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > > The purpose of this child device is to provide shared
> > > > memory management for exchanging tuning data between co-processor
> > > > and the Pass 1 unit of the camera ISP system, including cache
> > > > buffer handling.
> > > >
> > >
> > > Looks like we haven't really progressed on getting this replaced with
> > > something that doesn't require so much custom code. Let me propose something
> > > better then.
> > >
> > > We already have a reserved memory mode in DT. If it has a compatible string
> > > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > > [1]. That would make it available for consumer devices to look-up.
> > >
> > > Now if we add a "memory-region" property to the SCP device node and point it
> > > to our reserved memory node, the SCP driver could look it up and hook to the
> > > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> > >
> > > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > > struct device use the coherent DMA ops, which operate on the assigned memory
> > > pool. With that, the P1 driver could just directly use those calls to
> > > manage the memory, without any custom code.
> > >
> > > There is an example how this setup works in the s5p-mfc driver[3], but it
> > > needs to be noted that it creates child nodes, because it can have more than
> > > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > > need child nodes and could just use the SCP device directly.
> > >
> > > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> > >
> > > Let me also post some specific comments below, in case we end up still
> > > needing any of the code.
> > >
> >
> > Thanks your suggestions.
> >
> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 

Ok, we will rename this.

> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

Got it. We will revise to use dma_map_single() with ptr.

> >                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
> >                                   DMA_ATTR_SKIP_CPU_SYNC);
> >         if (dma_mapping_error(dev, addr)) {
> >                 isp_ctx->scp_mem_pa = 0;
> 
> We also need to free the allocated memory.
> 

Ok, we will add the dma_unmap_single to free the allocated memory.

> >                 dev_err(dev, "Failed to map scp iova\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_iova = addr;
> >
> > Moreover, we have another meta input buffer usage.
> > For this kind of buffer, it will be allocated by V4L2 framework
> > with dma_alloc_coherent with SCP device. In order to get IOVA,
> > we will add dma_map_page_attrs in vb2_ops' buf_init function.
> > In buf_cleanup function, we will call dma_unmap_page_attrs function.
> 
> As per above, we don't have access to the struct page we want to map.
> We probably want to get the CPU VA using vb2_plane_vaddr() and call
> dma_map_single() instead.
> 

Got it. We will revise this to use dma_map_single() with CPU VA which is
got from vb2_plane_vaddr() function.

> >
> > Based on these current implementation, do you think it is correct?
> > If we got any wrong, please let us know.
> >
> > Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> > avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> > Maybe we could not get the correct sg_table.
> > Do you think it is a bug and need to fix?
> 
> I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
> that don't need to be accessed from the kernel anyway, to avoid
> unnecessary kernel mapping operations. However, for coherent memory
> pool, it doesn't change anything, because the memory always has a
> kernel mapping. We also need the kernel virtual address for
> dma_map_single(). Also the flag doesn't eliminate the need to do the
> sync, e.g. if the userspace accesses the buffer.
> 
> Could you give me more information about the failure you're seeing?
> Where is the dma_sync_sg_for_device() called from? Where do you get
> the sgtable from?
> 
> Best regards,
> Tomasz

Sorry. I forgot provide one information related to this issue.
Here is the call stack of panic KE if we enable DMA_ATTR_NON_CONSISTENT
DMA flag. Maybe we should not enable this flag for coherent memory pool.

[Function]
vb2_dc_alloc

[Code]
	if (!(buf->attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
	    (buf->attrs & DMA_ATTR_NON_CONSISTENT))
		buf->dma_sgt = vb2_dc_get_base_sgt(buf);

[KE]
[   59.234326] pstate: 80000005 (Nzcv daif -PAN -UAO)
[   59.234935] pc : __clean_dcache_area_poc+0x20/0x38
[   59.235537] lr : __swiotlb_sync_sg_for_device+0x74/0x9c
[   59.249430] Call trace:
[   59.249742]  __clean_dcache_area_poc+0x20/0x38
[   59.250303]  vb2_dc_prepare+0x5c/0x6c
[   59.250763]  __buf_prepare+0x790/0x8a4
[   59.251234]  vb2_req_prepare+0x38/0x68
[   59.251707]  vb2_request_validate+0x40/0x9c
[   59.252235]  media_request_ioctl+0x124/0x2a4
[   59.252774]  __arm64_compat_sys_ioctl+0xf4/0x25c
[   59.253356]  el0_svc_common+0xa4/0x154
[   59.253828]  el0_svc_compat_handler+0x2c/0x38
[   59.254377]  el0_svc_compat+0x8/0x18
[   59.254827] Code: 9ac32042 8b010001 d1000443 8a230000 (d50b7a20)
[   59.255592] ---[ end trace eb37ebade032c2fc ]---
[   59.256173] Kernel panic - not syncing: Fatal exception

Thanks,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  5:44               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  5:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:
On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz,
> >
> > On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > > The purpose of this child device is to provide shared
> > > > memory management for exchanging tuning data between co-processor
> > > > and the Pass 1 unit of the camera ISP system, including cache
> > > > buffer handling.
> > > >
> > >
> > > Looks like we haven't really progressed on getting this replaced with
> > > something that doesn't require so much custom code. Let me propose something
> > > better then.
> > >
> > > We already have a reserved memory mode in DT. If it has a compatible string
> > > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > > [1]. That would make it available for consumer devices to look-up.
> > >
> > > Now if we add a "memory-region" property to the SCP device node and point it
> > > to our reserved memory node, the SCP driver could look it up and hook to the
> > > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> > >
> > > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > > struct device use the coherent DMA ops, which operate on the assigned memory
> > > pool. With that, the P1 driver could just directly use those calls to
> > > manage the memory, without any custom code.
> > >
> > > There is an example how this setup works in the s5p-mfc driver[3], but it
> > > needs to be noted that it creates child nodes, because it can have more than
> > > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > > need child nodes and could just use the SCP device directly.
> > >
> > > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> > >
> > > Let me also post some specific comments below, in case we end up still
> > > needing any of the code.
> > >
> >
> > Thanks your suggestions.
> >
> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 

Ok, we will rename this.

> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

Got it. We will revise to use dma_map_single() with ptr.

> >                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
> >                                   DMA_ATTR_SKIP_CPU_SYNC);
> >         if (dma_mapping_error(dev, addr)) {
> >                 isp_ctx->scp_mem_pa = 0;
> 
> We also need to free the allocated memory.
> 

Ok, we will add the dma_unmap_single to free the allocated memory.

> >                 dev_err(dev, "Failed to map scp iova\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_iova = addr;
> >
> > Moreover, we have another meta input buffer usage.
> > For this kind of buffer, it will be allocated by V4L2 framework
> > with dma_alloc_coherent with SCP device. In order to get IOVA,
> > we will add dma_map_page_attrs in vb2_ops' buf_init function.
> > In buf_cleanup function, we will call dma_unmap_page_attrs function.
> 
> As per above, we don't have access to the struct page we want to map.
> We probably want to get the CPU VA using vb2_plane_vaddr() and call
> dma_map_single() instead.
> 

Got it. We will revise this to use dma_map_single() with CPU VA which is
got from vb2_plane_vaddr() function.

> >
> > Based on these current implementation, do you think it is correct?
> > If we got any wrong, please let us know.
> >
> > Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> > avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> > Maybe we could not get the correct sg_table.
> > Do you think it is a bug and need to fix?
> 
> I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
> that don't need to be accessed from the kernel anyway, to avoid
> unnecessary kernel mapping operations. However, for coherent memory
> pool, it doesn't change anything, because the memory always has a
> kernel mapping. We also need the kernel virtual address for
> dma_map_single(). Also the flag doesn't eliminate the need to do the
> sync, e.g. if the userspace accesses the buffer.
> 
> Could you give me more information about the failure you're seeing?
> Where is the dma_sync_sg_for_device() called from? Where do you get
> the sgtable from?
> 
> Best regards,
> Tomasz

Sorry. I forgot provide one information related to this issue.
Here is the call stack of panic KE if we enable DMA_ATTR_NON_CONSISTENT
DMA flag. Maybe we should not enable this flag for coherent memory pool.

[Function]
vb2_dc_alloc

[Code]
	if (!(buf->attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
	    (buf->attrs & DMA_ATTR_NON_CONSISTENT))
		buf->dma_sgt = vb2_dc_get_base_sgt(buf);

[KE]
[   59.234326] pstate: 80000005 (Nzcv daif -PAN -UAO)
[   59.234935] pc : __clean_dcache_area_poc+0x20/0x38
[   59.235537] lr : __swiotlb_sync_sg_for_device+0x74/0x9c
[   59.249430] Call trace:
[   59.249742]  __clean_dcache_area_poc+0x20/0x38
[   59.250303]  vb2_dc_prepare+0x5c/0x6c
[   59.250763]  __buf_prepare+0x790/0x8a4
[   59.251234]  vb2_req_prepare+0x38/0x68
[   59.251707]  vb2_request_validate+0x40/0x9c
[   59.252235]  media_request_ioctl+0x124/0x2a4
[   59.252774]  __arm64_compat_sys_ioctl+0xf4/0x25c
[   59.253356]  el0_svc_common+0xa4/0x154
[   59.253828]  el0_svc_compat_handler+0x2c/0x38
[   59.254377]  el0_svc_compat+0x8/0x18
[   59.254827] Code: 9ac32042 8b010001 d1000443 8a230000 (d50b7a20)
[   59.255592] ---[ end trace eb37ebade032c2fc ]---
[   59.256173] Kernel panic - not syncing: Fatal exception

Thanks,

Jungo



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  5:44               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  5:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:
On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz,
> >
> > On Mon, 2019-07-01 at 16:25 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:44AM +0800, Jungo Lin wrote:
> > > > The purpose of this child device is to provide shared
> > > > memory management for exchanging tuning data between co-processor
> > > > and the Pass 1 unit of the camera ISP system, including cache
> > > > buffer handling.
> > > >
> > >
> > > Looks like we haven't really progressed on getting this replaced with
> > > something that doesn't require so much custom code. Let me propose something
> > > better then.
> > >
> > > We already have a reserved memory mode in DT. If it has a compatible string
> > > of "shared-dma-pool", it would be registered in the coherent DMA framework
> > > [1]. That would make it available for consumer devices to look-up.
> > >
> > > Now if we add a "memory-region" property to the SCP device node and point it
> > > to our reserved memory node, the SCP driver could look it up and hook to the
> > > DMA mapping API using of_reserved_mem_device_init_by_idx[2].
> > >
> > > That basically makes any dma_alloc_*(), dma_map_*(), etc. calls on the SCP
> > > struct device use the coherent DMA ops, which operate on the assigned memory
> > > pool. With that, the P1 driver could just directly use those calls to
> > > manage the memory, without any custom code.
> > >
> > > There is an example how this setup works in the s5p-mfc driver[3], but it
> > > needs to be noted that it creates child nodes, because it can have more than
> > > 1 DMA port, which may need its own memory pool. In our case, we wouldn't
> > > need child nodes and could just use the SCP device directly.
> > >
> > > [1] https://elixir.bootlin.com/linux/v5.2-rc7/source/kernel/dma/coherent.c#L345
> > > [2] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/of/of_reserved_mem.c#L312
> > > [3] https://elixir.bootlin.com/linux/v5.2-rc7/source/drivers/media/platform/s5p-mfc/s5p_mfc.c#L1075
> > >
> > > Let me also post some specific comments below, in case we end up still
> > > needing any of the code.
> > >
> >
> > Thanks your suggestions.
> >
> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 

Ok, we will rename this.

> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

Got it. We will revise to use dma_map_single() with ptr.

> >                                   MAX_COMPOSER_SIZE, DMA_BIDIRECTIONAL,
> >                                   DMA_ATTR_SKIP_CPU_SYNC);
> >         if (dma_mapping_error(dev, addr)) {
> >                 isp_ctx->scp_mem_pa = 0;
> 
> We also need to free the allocated memory.
> 

Ok, we will add the dma_unmap_single to free the allocated memory.

> >                 dev_err(dev, "Failed to map scp iova\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_iova = addr;
> >
> > Moreover, we have another meta input buffer usage.
> > For this kind of buffer, it will be allocated by V4L2 framework
> > with dma_alloc_coherent with SCP device. In order to get IOVA,
> > we will add dma_map_page_attrs in vb2_ops' buf_init function.
> > In buf_cleanup function, we will call dma_unmap_page_attrs function.
> 
> As per above, we don't have access to the struct page we want to map.
> We probably want to get the CPU VA using vb2_plane_vaddr() and call
> dma_map_single() instead.
> 

Got it. We will revise this to use dma_map_single() with CPU VA which is
got from vb2_plane_vaddr() function.

> >
> > Based on these current implementation, do you think it is correct?
> > If we got any wrong, please let us know.
> >
> > Btw, we also DMA_ATTR_NO_KERNEL_MAPPING DMA attribte to
> > avoid dma_sync_sg_for_device. Othewise, it will hit the KE.
> > Maybe we could not get the correct sg_table.
> > Do you think it is a bug and need to fix?
> 
> I think DMA_ATTR_NO_KERNEL_MAPPING is good to have for all the buffers
> that don't need to be accessed from the kernel anyway, to avoid
> unnecessary kernel mapping operations. However, for coherent memory
> pool, it doesn't change anything, because the memory always has a
> kernel mapping. We also need the kernel virtual address for
> dma_map_single(). Also the flag doesn't eliminate the need to do the
> sync, e.g. if the userspace accesses the buffer.
> 
> Could you give me more information about the failure you're seeing?
> Where is the dma_sync_sg_for_device() called from? Where do you get
> the sgtable from?
> 
> Best regards,
> Tomasz

Sorry. I forgot provide one information related to this issue.
Here is the call stack of panic KE if we enable DMA_ATTR_NON_CONSISTENT
DMA flag. Maybe we should not enable this flag for coherent memory pool.

[Function]
vb2_dc_alloc

[Code]
	if (!(buf->attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
	    (buf->attrs & DMA_ATTR_NON_CONSISTENT))
		buf->dma_sgt = vb2_dc_get_base_sgt(buf);

[KE]
[   59.234326] pstate: 80000005 (Nzcv daif -PAN -UAO)
[   59.234935] pc : __clean_dcache_area_poc+0x20/0x38
[   59.235537] lr : __swiotlb_sync_sg_for_device+0x74/0x9c
[   59.249430] Call trace:
[   59.249742]  __clean_dcache_area_poc+0x20/0x38
[   59.250303]  vb2_dc_prepare+0x5c/0x6c
[   59.250763]  __buf_prepare+0x790/0x8a4
[   59.251234]  vb2_req_prepare+0x38/0x68
[   59.251707]  vb2_request_validate+0x40/0x9c
[   59.252235]  media_request_ioctl+0x124/0x2a4
[   59.252774]  __arm64_compat_sys_ioctl+0xf4/0x25c
[   59.253356]  el0_svc_common+0xa4/0x154
[   59.253828]  el0_svc_compat_handler+0x2c/0x38
[   59.254377]  el0_svc_compat+0x8/0x18
[   59.254827] Code: 9ac32042 8b010001 d1000443 8a230000 (d50b7a20)
[   59.255592] ---[ end trace eb37ebade032c2fc ]---
[   59.256173] Kernel panic - not syncing: Fatal exception

Thanks,

Jungo



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-05  4:22           ` [RFC,v3 " Tomasz Figa
  (?)
@ 2019-07-05  7:59               ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  7:59 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Frederic Chen (陳俊元),
	list-Y9sIeH5OGRo

Hi Tomasz:

On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> >
> > Hi Tomasz,

[snip]

> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 
> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

We have changed to use ma_map_single() with ptr, but encounter IOMMU
error. From the debug log of iommu_dma_map_page[3], we got
0x0000000054800000 instead of expected address: 0x0000000050800000[2].
There is a address offset(0x4000000). If we change to use
dma_map_page_attrs with phys_to_page(addr), the address is correct as we
expected[2]. Do you have any suggestion on this issue? Do we miss
something?

[1]
[    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000

[    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000

[    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
[    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000

[2]
[    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000
[    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000
[    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
[    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000


[3]
dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
		unsigned long offset, size_t size, int prot)
{
	phys_addr_t phys = page_to_phys(page);
	pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);

	return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
			iommu_get_dma_domain(dev));
}

[snip]

Best regards,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  7:59               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  7:59 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi Tomasz:

On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz,

[snip]

> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 
> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

We have changed to use ma_map_single() with ptr, but encounter IOMMU
error. From the debug log of iommu_dma_map_page[3], we got
0x0000000054800000 instead of expected address: 0x0000000050800000[2].
There is a address offset(0x4000000). If we change to use
dma_map_page_attrs with phys_to_page(addr), the address is correct as we
expected[2]. Do you have any suggestion on this issue? Do we miss
something?

[1]
[    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000

[    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000

[    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
[    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000

[2]
[    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000
[    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000
[    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
[    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000


[3]
dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
		unsigned long offset, size_t size, int prot)
{
	phys_addr_t phys = page_to_phys(page);
	pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);

	return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
			iommu_get_dma_domain(dev));
}

[snip]

Best regards,

Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-05  7:59               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-05  7:59 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi Tomasz:

On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz,

[snip]

> > After applying your suggestion in SCP device driver, we could remove
> > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > to get SCP address. We could touch the buffer with this SCP address in
> > SCP processor.
> >
> > After that, we use dma_map_page_attrs with P1 device which supports
> > IOMMU domain to get IOVA address. For this address, we will assign
> > it to our ISP HW device to proceed.
> >
> > Below is the snippet for ISP P1 compose buffer initialization.
> >
> >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> >         if (!ptr) {
> >                 dev_err(dev, "failed to allocate compose memory\n");
> >                 return -ENOMEM;
> >         }
> >         isp_ctx->scp_mem_pa = addr;
> 
> addr contains a DMA address, not a physical address. Could we call it
> scp_mem_dma instead?
> 
> >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> >
> >         /* get iova address */
> >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> 
> addr is a DMA address, so phys_to_page() can't be called on it. The
> simplest thing here would be to use dma_map_single() with ptr as the
> CPU address expected.
> 

We have changed to use ma_map_single() with ptr, but encounter IOMMU
error. From the debug log of iommu_dma_map_page[3], we got
0x0000000054800000 instead of expected address: 0x0000000050800000[2].
There is a address offset(0x4000000). If we change to use
dma_map_page_attrs with phys_to_page(addr), the address is correct as we
expected[2]. Do you have any suggestion on this issue? Do we miss
something?

[1]
[    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000

[    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000

[    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
[    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000

[2]
[    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
device_base:0x0000000050000000 dma:0x0000000050800000
virt_base:ffffff8014000000 va:ffffff8014800000
[    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
va:ffffff8014800000
[    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
[    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000


[3]
dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
		unsigned long offset, size_t size, int prot)
{
	phys_addr_t phys = page_to_phys(page);
	pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);

	return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
			iommu_get_dma_domain(dev));
}

[snip]

Best regards,

Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-06-11  3:53     ` Jungo Lin
  (?)
@ 2019-07-10  9:54       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and seven video devices during
> initialization. Moreover, it also connects with sensor and
> seninf drivers with V4L2 async APIs.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "media: support Mediatek sensor interface driver"[1].
> 
> ISP P1 sub-device communicates with seninf sub-device with CIO.
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/10979135/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
>  3 files changed, 1848 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> 

Thanks for the patch. Please see my comments inline.

[snip]

> +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> +				      struct media_request *new_req)
> +{
> +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> +
> +	if (!cam_dev->streaming) {
> +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> +		spin_lock(&cam_dev->req_lock);
> +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> +		spin_unlock(&cam_dev->req_lock);
> +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> +		return;
> +	}
> +
> +	/* Normal enqueue flow */
> +	if (new_req) {
> +		mtk_isp_req_enqueue(dev, new_req);
> +		return;
> +	}
> +
> +	/* Flush all media requests wehen first stream on */
> +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> +		list_del(&req->list);
> +		mtk_isp_req_enqueue(dev, &req->req);
> +	}
> +}

This will have to be redone, as per the other suggestions, but generally one
would have a function that tries to queue as much as possible from a list to
the hardware and another function that adds a request to the list and calls
the first function.

> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> +
> +	vb2_request_queue(req);
> +	mtk_cam_req_try_isp_queue(cam_dev, req);

Looks like this driver is suffering from versy similar problems in request
handling as the DIP driver used to.

I'd prefer to save my time and avoid repeating the same comments, so please
check my comments for the DIP driver and apply them to this one too:

https://patchwork.kernel.org/patch/10905223/

> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)

Doesn't this function return bits not bytes?

> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> +					__u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);

The __ prefixed types should be used only inside UAPI. Please change the
driver to use the normal ones.

> +
> +	width = ALIGN(width, 4);

If there is some alignment requirement for width, it should be handled by
TRY_/S_FMT and here we should already assume everything properly aligned.

> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +
> +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> +				       __u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);

Ditto.

> +	stride = DIV_ROUND_UP(width * 3, 2);

Could we introduce a local variable for this intermediate value, so that its
name could explain what the value is?

> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +
> +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> +		stride = ALIGN(stride, 4);

Is it expected that only the F10 format needs this alignment?

> +
> +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Calcuate mplane pix format */
> +static void
> +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> +			   struct v4l2_pix_format_mplane *dest_fmt,
> +			   unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;

Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?

> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = img_cal_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));

This memset is not needed. The core clears the reserved fields
automatically:

https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559

(We may want to backport the patch that added that to our 4.19 branch.)

> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;

Hmm, we only seem to support 1 plane raw formats in this driver. Does the
hardware support any formats with more than 1 plane? If not, all the code
should be simplified to just assume 1 plane.

> +}
> +
> +static void
> +mtk_cam_dev_set_img_fmt(struct device *dev,
> +			struct v4l2_pix_format_mplane *dest_fmt,
> +			const struct v4l2_pix_format_mplane *src_fmt,
> +			unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));

Given that src_fmt should already be validated and have any fields adjusted
to match the driver requirements, wouldn't all the lines above be equivalent
to *dest_fmt = *src_fmt?

We probably want to move setting all the constant fields to
mtk_cam_vidioc_try_fmt().

> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);

This should have been called already before this function was called,
because src_fmt should be already expected to contain valid settings. In
fact, this is already called in mtk_cam_vidioc_try_fmt().

> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct device *dev,

Please don't pass struct device pointer around, but instead just the main
driver data struct, which should be much more convenient for accessing
various driver data. Please fix the other functions as well.

> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_set_img_fmt(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);

We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
v4l2_format struct and have any incorrect fields replaced by
mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
v4l2_format at runtime.

> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +static int mtk_cam_isp_open(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	int ret;
> +
> +	mutex_lock(&cam_dev->lock);
> +	ret = v4l2_fh_open(file);
> +	if (ret)
> +		goto unlock;
> +
> +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);

Please don't power on open. Normally applications keep the device nodes open
all the time, so they would keep everything powered on.

Normally this should be done as late as possible, ideally when starting the
streaming.

> +	if (ret)
> +		dev_err(dev, "%s fail:%d", __func__, ret);
> +
> +unlock:
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_isp_release(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	mutex_lock(&cam_dev->lock);
> +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> +	vb2_fop_release(file);
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return 0;
> +}

If we remove power handling from open and release, we should be able to just
use v4l2_fh_open() and vb2_fop_release() directly in the
v4l2_file_operations struct.

> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct v4l2_subdev *sensor;

This variable would be unitialized if there is no streaming sensor. Was
there no compiler warning generated for this?

> +
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "Sensor is not connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;

I don't think -EPERM is a good error code here. It's about a missing seninf
device, so perhaps -ENODEV?

> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> +	if (!cam_dev->sensor)
> +		return -EPERM;

> -ENODEV

> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;

Maybe just return ret?

> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> +	isp_composer_stream(dev, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +
> +	return -EPERM;

return ret?

> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	isp_composer_stream(dev, 0);

Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
can't guarantee that the hardware releases all the memory that we're going
to free once this function returns.

> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> +
> +	return on ? mtk_isp_power_init(cam_dev) :
> +		    mtk_isp_power_release(&cam_dev->pdev->dev);

s_power is a historical thing and we shouldn't be implementing it. Instead,
we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
whenever we start and stop streaming respectively.

> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)

I assume this check is needed, because the pads with higher indexes are not
video nodes? If so, a comment would be helpful here.

> +		cam_dev->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);

This can be folded into the declaration.

> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		buf->vbb.request_fd,
> +		buf->vbb.vb2_buf.index);
> +
> +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> +	if (vb->vb2_queue->uses_requests)
> +		return;

I'd suggest removing non-request support from this driver. Even if we end up
with a need to provide compatibility for non-request mode, then it should be
built on top of the requests mode, so that the driver itself doesn't have to
deal with two modes.

> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *smem_dev = cam_dev->smem_dev;
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;

Just a reminder that this will have to be reworked according to my comments
for the memory allocation patch.

> +
> +	/* scp address is only valid for meta input buffer */
> +	if (node->desc.smem_alloc)
> +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> +							      buf->daddr);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size)
> +		return -EINVAL;

For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
Otherwise we could get not enough or invalid data.

> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);

This shouldn't be called on OUTPUT buffers.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {

We should also verify that *num_planes == 1, as we don't support more
planes in this driver.

> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct mtk_cam_dev_request *req, *req0;
> +	struct media_request_object *obj, *obj0;
> +	struct vb2_buffer *vb;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {

nit: One would normally call the second argument "prev", or "b_prev".

> +		vb = &b->vbb.vb2_buf;
> +		if (vb->state == VB2_BUF_STATE_ACTIVE)

We shouldn't need to check the buffer state.

> +			vb2_buffer_done(vb, state);
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	spin_lock(&cam_dev->req_lock);
> +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {

nit: Ditto.

> +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {

Need to check if the object is a buffer.

> +			vb = container_of(obj, struct vb2_buffer, req_obj);
> +			if (vb->state == VB2_BUF_STATE_ACTIVE)

vb->state shouldn't be accessed directly from the drivers.

Generally, the need to check the state here would suggest that there is
something wrong with how the driver manages the requests. The list that is
being iterated here shouldn't contain any requests that have buffers that
aren't active. That will be achieved if my comments for the request handling
in the DIP driver are applied to this driver as well.

> +				vb2_buffer_done(vb, state);
> +		}
> +		list_del(&req->list);
> +	}
> +	spin_unlock(&cam_dev->req_lock);
> +
> +	if (node->vbq.uses_requests)
> +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> +	int ret;
> +
> +	if (!node->enabled) {

How is this synchronized with mtk_cam_media_link_setup()?

> +		dev_err(dev, "Node:%d is not enable\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_no_link;
> +	}
> +
> +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> +		atomic_read(&cam_dev->streamed_node_count), node_count);
> +
> +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> +		return 0;

How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
between calls to this function on different video nodes?

> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret) {
> +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> +		goto fail_start_pipeline;
> +	}
> +

Related to the above comment: If we start the media pipeline when we start
streaming on the first node, we would naturally prevent the link
configuration changes until the last node stops streaming (as long as the
link is not DYNAMIC). Note that it would only mark the entities as
streaming, but it wouldn't call their s_stream, which I believe is exactly
what we would need to solve the problem above.

> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> +		goto fail_stream_on;
> +	}
> +
> +	return 0;
> +
> +fail_stream_on:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_start_pipeline:
> +	atomic_dec(&cam_dev->streamed_node_count);
> +fail_no_link:
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	if (!node->enabled)
> +		return;

It shouldn't be possible for this to happen, because nobody could have
called start_streaming on a disabled node.

> +
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);

Shouldn't we stop streaming first, so that the hardware operation is
cancelled and any buffers owned by the hardware are released?

> +
> +	dev_dbg(dev, "%s: count info:%d", __func__,
> +		cam_dev->subdev.entity.stream_count);
> +
> +	/* Check the first node to stream-off */
> +	if (!cam_dev->subdev.entity.stream_count)
> +		return;
> +
> +	media_pipeline_stop(&node->vdev.entity);
> +
> +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +		dev_err(dev, "failed to stop streaming\n");
> +}
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);

This would end up being called multiple times, once for each video node.
Instead, this should be called explicitly by the driver when it completed
the request - perhaps in the frame completion handler?

With that, we probably wouldn't even need this callback.

> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));

We could just use dev_driver_name(cam_dev->dev) for both.

> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));

We should just store dev in cam_dev.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;

Is the set of formats available always the same regardless of the sensor
format?

> +	f->flags = 0;

We need f->description too.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (!node->desc.num_fmts)
> +		return -EINVAL;

When would that condition happen?

> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	const struct v4l2_format *dev_fmt;
> +	__u32  width, height;

Don't use __ types in implementation, they are here for UAPI purposes. There
is u32, which you could use instead, but for width and height you don't need
explicit size, so unsigned int should be good.

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc, in_fmt);

We shouldn't just load a default format. This function should validate all
the fields one by one and adjust them to something appropriate.

> +	}

CodingStyle: No braces if both if and else bodies have only 1 statement
each.

> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);

Shouldn't we clamp this with some maximum value too?

> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);

Ditto.

> +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> +				   &in_fmt->fmt.pix_mp, node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;

I think this should rather be something like vb2_queue_is_busy(), which
would prevent format changes if buffers are allocated.

> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +				     struct v4l2_input *input)
> +{
> +	if (input->index)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> +				  unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_input(struct file *file,
> +				  void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +	.s_power = mtk_cam_sd_s_power,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {

nit: mtk_cam_media_entity_ops?

> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = mtk_cam_isp_open,
> +	.release = mtk_cam_isp_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {

nit: Those are media ops, so perhaps just mtk_cam_media_ops?

> +	.link_notify = v4l2_pipeline_link_notify,
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,

Could we replace any use of this macro with dev_driver_string(dev) and then
delete the macro? The less name strings in the driver the better, as there
is less change for confusing the userspace if few different names are used
at the same time.

> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +
> +	return media_device_register(media_dev);
> +}
> +
> +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];

Would it make sense to pass node as an argument to this function instead of
(or in addition to) i?

> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;

Why not call it capture instead and avoid the inversion?

> +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> +	int ret;
> +
> +	cam_dev->subdev_pads[i].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->enabled = false;

Are all the nodes optional? If there is any required node, it should be
always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.

> +	node->id = i;
> +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;

Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?

> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     &node->desc,
> +				     &node->vdev_fmt);
> +
> +	/* Initialize vbq */
> +	vbq->type = node->vdev_fmt.type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam_dev->smem_dev;
> +	} else {
> +		vbq->dev = &cam_dev->pdev->dev;
> +	}
> +
> +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		vdev->entity.function =
> +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;

This is a video node, so it's just a DMA, not a processing entity. I believe
all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.

> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +	/* Put the process hub sub device in the vb2 private data */

What is "process hub" and what "sub device" is this about?

> +	vbq->drv_priv = cam_dev;
> +	vbq->lock = &node->lock;
> +	vbq->supports_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_vb2_queue;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->lock;
> +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.ops = NULL;
> +	/* Enable private control for image video devices */
> +	if (node->desc.image) {
> +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +		vdev->ctrl_handler = &node->ctrl_handler;
> +	}
> +	video_set_drvdata(vdev, cam_dev);
> +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vdev;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam_dev->subdev.entity,
> +					    i, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> +					    i, &vdev->entity, 0,
> +					    link_flags);
> +	}
> +	if (ret)
> +		goto fail_link;
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);
> +	INIT_LIST_HEAD(&node->pending_list);

This should be all initialized before registering the video device.
Otherwise userspace could open the device before these are initialized.

> +
> +	return 0;
> +
> +fail_link:
> +	video_unregister_device(vdev);
> +fail_vdev:
> +	vb2_queue_release(vbq);
> +fail_vb2_queue:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)

This function doesn't have anything to do with mem2mem. How about
mtk_cam_v4l2_register()?

Perhaps it would make sense to move any media related code into into
mtk_cam_media_register(), keep only V4L2 related code here and have the
caller call the former first and then this one, rather than having such deep
sequence of nested calls, which makes the driver harder to read.

> +{
> +	struct device *dev = &cam_dev->pdev->dev;

How about just storing dev, instead of pdev in the struct? Also, calling the
argument "cam", would make it as short as cam->dev.

> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);

An info message should be useful to the user in some way. Printing kernel
pointers isn't useful. Something like "registered media0" could be useful to
let the user know which media device is associated with this driver if there
is more than one in the system.

> +
> +	/* Set up v4l2 device */
> +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;

Please call the labels after the cleanup step that needs to be done. It
makes it easier to spot any ordering errors.

> +	}
> +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);

Same as above.

> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);
> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);

Stray ":" at the end of the message.

> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		ret = mtk_cam_video_register_device(cam_dev, i);
> +		if (ret)
> +			goto fail_video_register;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_video_register:
> +	i--;

This could be moved into the for clause, as the initialization statement.

> +	for (; i >= 0; i--) {

i is unsigned. Did this compile without warnings?

> +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);

Should we move this into mtk_cam_video_unregister_device() to be consistent
with registration?

> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);

Please print errors only where they actually happen, not at the cleanup.

> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;

nit: Move the declaration inside the for loop, since the variable is only
used there.

> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		dev = &cam_dev->vdev_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);

This was allocated using devm_kcalloc(), so no need to free it explicitly.

> +
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_CIO_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_CIO_PAD_SINK,
> +				    0);
> +	if (ret) {
> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");

dev_dbg()

> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}

Why not just put the contents of this function inside 
mtk_cam_dev_notifier_complete()?

> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +

Should we somehow check that the entity we got is seninf indeed and there
was no mistake in DT?

> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);

bound

Also please make this dev_dbg().

> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = NULL;
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);

unbound

> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> +		NULL);
> +	if (ret)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;

Could we print some error messages for the 2 cases above?

> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");

dev_dbg()

> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,

I don't think we need vidioc_*_input. At least the current implementation in
this patch doesn't seem to do anything useful.

> +	/* buffer queue management */

Drop this comment, as it's obvious that the callbacks with "buf" in the name
are related to buffers, there are some non-buffer callbacks below too (e.g.
vidioc_subscribe_event) and also the other ops structs below don't have such
comment.

> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,

PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
making it 512 * SZ_1K?

That said, it should normally be just sizeof(struct some_struct_used_here).

Same for the other entries below.

> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = IMG_MAX_WIDTH,
> +			.min_width = IMG_MIN_WIDTH,
> +			.max_height = IMG_MAX_HEIGHT,
> +			.min_height = IMG_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = RRZ_MAX_WIDTH,
> +			.min_width = RRZ_MIN_WIDTH,
> +			.max_height = RRZ_MAX_HEIGHT,
> +			.min_height = RRZ_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 1,

Why not just make it always 0 and move the default format to the beginning
of stream_out_fmts[]?

> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[0],

nit: If you only need to point to these data once, you could define them in
place, as a compound literal, like

>                 .frmsizes = &(struct v4l2_framesizeenum) {
>                         // initialize here
>                 },

> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[1],
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,

link_flags seems to be 0 for all nodes.

> +		.capture = true,

We already know this from .buf_type. We can check using the
V4L2_TYPE_IS_OUTPUT() macro.

> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),

I don't think this is correct. The description suggests one specific format
(local motion vector histogram), so the queue should probably be hardwired
to that format?

> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,

Where does this number come from?

> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)

< ARRAY_SIZE(output_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)

< ARRAY_SIZE(capture_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;

Do we need this additional mtk_cam_dev struct? Couldn't we just use
mtk_isp_p1 here?

> +	mtk_cam_dev_queue_setup(cam_dev);
> +	/* v4l2 sub-device registration */
> +
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);

This debugging message doesn't seem very useful. Please remove.

> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret) {
> +		mtk_cam_v4l2_unregister(cam_dev);

Please use an error path on the bottom of the function instead. With goto
labels named after the next clean-up step that needs to be performed.

> +		return ret;
> +	}
> +
> +	spin_lock_init(&cam_dev->req_lock);
> +	INIT_LIST_HEAD(&cam_dev->req_list);
> +	mutex_init(&cam_dev->lock);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,

"release" is normally used for file_operations. How about "cleanup"?

> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	mutex_destroy(&cam_dev->lock);
> +
> +	return 0;
> +}

I'd suggest moving any generic API implementation code (platform_device,
V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
mtk_cam_hw.c.

> +
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"

Maybe it's not a critical thing, but generally it's a good practice to just
explicitly specific this name somewhere, e.g. struct
platform_driver::driver::name and then just refer to dev_driver_name(). It
makes it easier to make sure that the name stays the same everywhere.

> +
> +#define MTK_CAM_P1_META_IN_0			0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1

Since these are just some utility definitions, we can use enum instead of
assigning the values by hand.

> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> +#define MTK_CAM_P1_META_OUT_0			3
> +#define MTK_CAM_P1_META_OUT_1			4
> +#define MTK_CAM_P1_META_OUT_2			5
> +#define MTK_CAM_P1_META_OUT_3			6
> +#define MTK_CAM_P1_TOTAL_CAPTURE		6

Ditto.

> +
> +#define MTK_CAM_P1_TOTAL_NODES			7

Please just add the two totals together rather than manually specifying the
value.

> +
> +struct mtk_cam_dev_request {
> +	struct media_request	req;
> +	struct list_head	list;
> +};
> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +	unsigned int		node_id;
> +};

Could you add kerneldoc comments for the 2 structs?

> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + * @frmsizes:	supported frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device

vdev_pad?

> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *

Please fix the order of the documentation to match the order of the struct.

> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct video_device vdev;

Not documented above.

> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	/* protect for pending_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t slock;

How about calling it pending_list_lock?

> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @lock:       The mutex protecting video device open/release operations
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> + * @req_list:	Lins to keep media requests before streaming on
> + * @req_lock:	Protect the req_list data
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->

This probably isn't the best place for graph topology description. I think
we actually want a separate documentation file for this, similar to
Documentation/media/v4l-drivers/ipu3.rst.

> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	/* protect video device open/release operations */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	unsigned int streaming:1;
> +	atomic_t streamed_node_count;
> +	struct list_head req_list;
> +	/* protect for req_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t req_lock;

How about calling it req_list_lock?

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-10  9:54       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and seven video devices during
> initialization. Moreover, it also connects with sensor and
> seninf drivers with V4L2 async APIs.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "media: support Mediatek sensor interface driver"[1].
> 
> ISP P1 sub-device communicates with seninf sub-device with CIO.
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/10979135/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
>  3 files changed, 1848 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> 

Thanks for the patch. Please see my comments inline.

[snip]

> +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> +				      struct media_request *new_req)
> +{
> +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> +
> +	if (!cam_dev->streaming) {
> +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> +		spin_lock(&cam_dev->req_lock);
> +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> +		spin_unlock(&cam_dev->req_lock);
> +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> +		return;
> +	}
> +
> +	/* Normal enqueue flow */
> +	if (new_req) {
> +		mtk_isp_req_enqueue(dev, new_req);
> +		return;
> +	}
> +
> +	/* Flush all media requests wehen first stream on */
> +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> +		list_del(&req->list);
> +		mtk_isp_req_enqueue(dev, &req->req);
> +	}
> +}

This will have to be redone, as per the other suggestions, but generally one
would have a function that tries to queue as much as possible from a list to
the hardware and another function that adds a request to the list and calls
the first function.

> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> +
> +	vb2_request_queue(req);
> +	mtk_cam_req_try_isp_queue(cam_dev, req);

Looks like this driver is suffering from versy similar problems in request
handling as the DIP driver used to.

I'd prefer to save my time and avoid repeating the same comments, so please
check my comments for the DIP driver and apply them to this one too:

https://patchwork.kernel.org/patch/10905223/

> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)

Doesn't this function return bits not bytes?

> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> +					__u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);

The __ prefixed types should be used only inside UAPI. Please change the
driver to use the normal ones.

> +
> +	width = ALIGN(width, 4);

If there is some alignment requirement for width, it should be handled by
TRY_/S_FMT and here we should already assume everything properly aligned.

> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +
> +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> +				       __u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);

Ditto.

> +	stride = DIV_ROUND_UP(width * 3, 2);

Could we introduce a local variable for this intermediate value, so that its
name could explain what the value is?

> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +
> +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> +		stride = ALIGN(stride, 4);

Is it expected that only the F10 format needs this alignment?

> +
> +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Calcuate mplane pix format */
> +static void
> +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> +			   struct v4l2_pix_format_mplane *dest_fmt,
> +			   unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;

Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?

> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = img_cal_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));

This memset is not needed. The core clears the reserved fields
automatically:

https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559

(We may want to backport the patch that added that to our 4.19 branch.)

> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;

Hmm, we only seem to support 1 plane raw formats in this driver. Does the
hardware support any formats with more than 1 plane? If not, all the code
should be simplified to just assume 1 plane.

> +}
> +
> +static void
> +mtk_cam_dev_set_img_fmt(struct device *dev,
> +			struct v4l2_pix_format_mplane *dest_fmt,
> +			const struct v4l2_pix_format_mplane *src_fmt,
> +			unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));

Given that src_fmt should already be validated and have any fields adjusted
to match the driver requirements, wouldn't all the lines above be equivalent
to *dest_fmt = *src_fmt?

We probably want to move setting all the constant fields to
mtk_cam_vidioc_try_fmt().

> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);

This should have been called already before this function was called,
because src_fmt should be already expected to contain valid settings. In
fact, this is already called in mtk_cam_vidioc_try_fmt().

> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct device *dev,

Please don't pass struct device pointer around, but instead just the main
driver data struct, which should be much more convenient for accessing
various driver data. Please fix the other functions as well.

> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_set_img_fmt(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);

We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
v4l2_format struct and have any incorrect fields replaced by
mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
v4l2_format at runtime.

> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +static int mtk_cam_isp_open(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	int ret;
> +
> +	mutex_lock(&cam_dev->lock);
> +	ret = v4l2_fh_open(file);
> +	if (ret)
> +		goto unlock;
> +
> +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);

Please don't power on open. Normally applications keep the device nodes open
all the time, so they would keep everything powered on.

Normally this should be done as late as possible, ideally when starting the
streaming.

> +	if (ret)
> +		dev_err(dev, "%s fail:%d", __func__, ret);
> +
> +unlock:
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_isp_release(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	mutex_lock(&cam_dev->lock);
> +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> +	vb2_fop_release(file);
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return 0;
> +}

If we remove power handling from open and release, we should be able to just
use v4l2_fh_open() and vb2_fop_release() directly in the
v4l2_file_operations struct.

> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct v4l2_subdev *sensor;

This variable would be unitialized if there is no streaming sensor. Was
there no compiler warning generated for this?

> +
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "Sensor is not connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;

I don't think -EPERM is a good error code here. It's about a missing seninf
device, so perhaps -ENODEV?

> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> +	if (!cam_dev->sensor)
> +		return -EPERM;

> -ENODEV

> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;

Maybe just return ret?

> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> +	isp_composer_stream(dev, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +
> +	return -EPERM;

return ret?

> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	isp_composer_stream(dev, 0);

Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
can't guarantee that the hardware releases all the memory that we're going
to free once this function returns.

> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> +
> +	return on ? mtk_isp_power_init(cam_dev) :
> +		    mtk_isp_power_release(&cam_dev->pdev->dev);

s_power is a historical thing and we shouldn't be implementing it. Instead,
we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
whenever we start and stop streaming respectively.

> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)

I assume this check is needed, because the pads with higher indexes are not
video nodes? If so, a comment would be helpful here.

> +		cam_dev->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);

This can be folded into the declaration.

> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		buf->vbb.request_fd,
> +		buf->vbb.vb2_buf.index);
> +
> +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> +	if (vb->vb2_queue->uses_requests)
> +		return;

I'd suggest removing non-request support from this driver. Even if we end up
with a need to provide compatibility for non-request mode, then it should be
built on top of the requests mode, so that the driver itself doesn't have to
deal with two modes.

> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *smem_dev = cam_dev->smem_dev;
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;

Just a reminder that this will have to be reworked according to my comments
for the memory allocation patch.

> +
> +	/* scp address is only valid for meta input buffer */
> +	if (node->desc.smem_alloc)
> +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> +							      buf->daddr);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size)
> +		return -EINVAL;

For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
Otherwise we could get not enough or invalid data.

> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);

This shouldn't be called on OUTPUT buffers.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {

We should also verify that *num_planes == 1, as we don't support more
planes in this driver.

> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct mtk_cam_dev_request *req, *req0;
> +	struct media_request_object *obj, *obj0;
> +	struct vb2_buffer *vb;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {

nit: One would normally call the second argument "prev", or "b_prev".

> +		vb = &b->vbb.vb2_buf;
> +		if (vb->state == VB2_BUF_STATE_ACTIVE)

We shouldn't need to check the buffer state.

> +			vb2_buffer_done(vb, state);
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	spin_lock(&cam_dev->req_lock);
> +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {

nit: Ditto.

> +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {

Need to check if the object is a buffer.

> +			vb = container_of(obj, struct vb2_buffer, req_obj);
> +			if (vb->state == VB2_BUF_STATE_ACTIVE)

vb->state shouldn't be accessed directly from the drivers.

Generally, the need to check the state here would suggest that there is
something wrong with how the driver manages the requests. The list that is
being iterated here shouldn't contain any requests that have buffers that
aren't active. That will be achieved if my comments for the request handling
in the DIP driver are applied to this driver as well.

> +				vb2_buffer_done(vb, state);
> +		}
> +		list_del(&req->list);
> +	}
> +	spin_unlock(&cam_dev->req_lock);
> +
> +	if (node->vbq.uses_requests)
> +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> +	int ret;
> +
> +	if (!node->enabled) {

How is this synchronized with mtk_cam_media_link_setup()?

> +		dev_err(dev, "Node:%d is not enable\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_no_link;
> +	}
> +
> +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> +		atomic_read(&cam_dev->streamed_node_count), node_count);
> +
> +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> +		return 0;

How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
between calls to this function on different video nodes?

> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret) {
> +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> +		goto fail_start_pipeline;
> +	}
> +

Related to the above comment: If we start the media pipeline when we start
streaming on the first node, we would naturally prevent the link
configuration changes until the last node stops streaming (as long as the
link is not DYNAMIC). Note that it would only mark the entities as
streaming, but it wouldn't call their s_stream, which I believe is exactly
what we would need to solve the problem above.

> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> +		goto fail_stream_on;
> +	}
> +
> +	return 0;
> +
> +fail_stream_on:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_start_pipeline:
> +	atomic_dec(&cam_dev->streamed_node_count);
> +fail_no_link:
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	if (!node->enabled)
> +		return;

It shouldn't be possible for this to happen, because nobody could have
called start_streaming on a disabled node.

> +
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);

Shouldn't we stop streaming first, so that the hardware operation is
cancelled and any buffers owned by the hardware are released?

> +
> +	dev_dbg(dev, "%s: count info:%d", __func__,
> +		cam_dev->subdev.entity.stream_count);
> +
> +	/* Check the first node to stream-off */
> +	if (!cam_dev->subdev.entity.stream_count)
> +		return;
> +
> +	media_pipeline_stop(&node->vdev.entity);
> +
> +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +		dev_err(dev, "failed to stop streaming\n");
> +}
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);

This would end up being called multiple times, once for each video node.
Instead, this should be called explicitly by the driver when it completed
the request - perhaps in the frame completion handler?

With that, we probably wouldn't even need this callback.

> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));

We could just use dev_driver_name(cam_dev->dev) for both.

> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));

We should just store dev in cam_dev.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;

Is the set of formats available always the same regardless of the sensor
format?

> +	f->flags = 0;

We need f->description too.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (!node->desc.num_fmts)
> +		return -EINVAL;

When would that condition happen?

> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	const struct v4l2_format *dev_fmt;
> +	__u32  width, height;

Don't use __ types in implementation, they are here for UAPI purposes. There
is u32, which you could use instead, but for width and height you don't need
explicit size, so unsigned int should be good.

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc, in_fmt);

We shouldn't just load a default format. This function should validate all
the fields one by one and adjust them to something appropriate.

> +	}

CodingStyle: No braces if both if and else bodies have only 1 statement
each.

> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);

Shouldn't we clamp this with some maximum value too?

> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);

Ditto.

> +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> +				   &in_fmt->fmt.pix_mp, node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;

I think this should rather be something like vb2_queue_is_busy(), which
would prevent format changes if buffers are allocated.

> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +				     struct v4l2_input *input)
> +{
> +	if (input->index)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> +				  unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_input(struct file *file,
> +				  void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +	.s_power = mtk_cam_sd_s_power,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {

nit: mtk_cam_media_entity_ops?

> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = mtk_cam_isp_open,
> +	.release = mtk_cam_isp_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {

nit: Those are media ops, so perhaps just mtk_cam_media_ops?

> +	.link_notify = v4l2_pipeline_link_notify,
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,

Could we replace any use of this macro with dev_driver_string(dev) and then
delete the macro? The less name strings in the driver the better, as there
is less change for confusing the userspace if few different names are used
at the same time.

> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +
> +	return media_device_register(media_dev);
> +}
> +
> +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];

Would it make sense to pass node as an argument to this function instead of
(or in addition to) i?

> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;

Why not call it capture instead and avoid the inversion?

> +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> +	int ret;
> +
> +	cam_dev->subdev_pads[i].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->enabled = false;

Are all the nodes optional? If there is any required node, it should be
always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.

> +	node->id = i;
> +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;

Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?

> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     &node->desc,
> +				     &node->vdev_fmt);
> +
> +	/* Initialize vbq */
> +	vbq->type = node->vdev_fmt.type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam_dev->smem_dev;
> +	} else {
> +		vbq->dev = &cam_dev->pdev->dev;
> +	}
> +
> +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		vdev->entity.function =
> +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;

This is a video node, so it's just a DMA, not a processing entity. I believe
all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.

> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +	/* Put the process hub sub device in the vb2 private data */

What is "process hub" and what "sub device" is this about?

> +	vbq->drv_priv = cam_dev;
> +	vbq->lock = &node->lock;
> +	vbq->supports_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_vb2_queue;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->lock;
> +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.ops = NULL;
> +	/* Enable private control for image video devices */
> +	if (node->desc.image) {
> +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +		vdev->ctrl_handler = &node->ctrl_handler;
> +	}
> +	video_set_drvdata(vdev, cam_dev);
> +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vdev;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam_dev->subdev.entity,
> +					    i, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> +					    i, &vdev->entity, 0,
> +					    link_flags);
> +	}
> +	if (ret)
> +		goto fail_link;
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);
> +	INIT_LIST_HEAD(&node->pending_list);

This should be all initialized before registering the video device.
Otherwise userspace could open the device before these are initialized.

> +
> +	return 0;
> +
> +fail_link:
> +	video_unregister_device(vdev);
> +fail_vdev:
> +	vb2_queue_release(vbq);
> +fail_vb2_queue:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)

This function doesn't have anything to do with mem2mem. How about
mtk_cam_v4l2_register()?

Perhaps it would make sense to move any media related code into into
mtk_cam_media_register(), keep only V4L2 related code here and have the
caller call the former first and then this one, rather than having such deep
sequence of nested calls, which makes the driver harder to read.

> +{
> +	struct device *dev = &cam_dev->pdev->dev;

How about just storing dev, instead of pdev in the struct? Also, calling the
argument "cam", would make it as short as cam->dev.

> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);

An info message should be useful to the user in some way. Printing kernel
pointers isn't useful. Something like "registered media0" could be useful to
let the user know which media device is associated with this driver if there
is more than one in the system.

> +
> +	/* Set up v4l2 device */
> +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;

Please call the labels after the cleanup step that needs to be done. It
makes it easier to spot any ordering errors.

> +	}
> +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);

Same as above.

> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);
> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);

Stray ":" at the end of the message.

> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		ret = mtk_cam_video_register_device(cam_dev, i);
> +		if (ret)
> +			goto fail_video_register;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_video_register:
> +	i--;

This could be moved into the for clause, as the initialization statement.

> +	for (; i >= 0; i--) {

i is unsigned. Did this compile without warnings?

> +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);

Should we move this into mtk_cam_video_unregister_device() to be consistent
with registration?

> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);

Please print errors only where they actually happen, not at the cleanup.

> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;

nit: Move the declaration inside the for loop, since the variable is only
used there.

> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		dev = &cam_dev->vdev_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);

This was allocated using devm_kcalloc(), so no need to free it explicitly.

> +
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_CIO_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_CIO_PAD_SINK,
> +				    0);
> +	if (ret) {
> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");

dev_dbg()

> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}

Why not just put the contents of this function inside 
mtk_cam_dev_notifier_complete()?

> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +

Should we somehow check that the entity we got is seninf indeed and there
was no mistake in DT?

> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);

bound

Also please make this dev_dbg().

> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = NULL;
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);

unbound

> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> +		NULL);
> +	if (ret)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;

Could we print some error messages for the 2 cases above?

> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");

dev_dbg()

> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,

I don't think we need vidioc_*_input. At least the current implementation in
this patch doesn't seem to do anything useful.

> +	/* buffer queue management */

Drop this comment, as it's obvious that the callbacks with "buf" in the name
are related to buffers, there are some non-buffer callbacks below too (e.g.
vidioc_subscribe_event) and also the other ops structs below don't have such
comment.

> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,

PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
making it 512 * SZ_1K?

That said, it should normally be just sizeof(struct some_struct_used_here).

Same for the other entries below.

> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = IMG_MAX_WIDTH,
> +			.min_width = IMG_MIN_WIDTH,
> +			.max_height = IMG_MAX_HEIGHT,
> +			.min_height = IMG_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = RRZ_MAX_WIDTH,
> +			.min_width = RRZ_MIN_WIDTH,
> +			.max_height = RRZ_MAX_HEIGHT,
> +			.min_height = RRZ_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 1,

Why not just make it always 0 and move the default format to the beginning
of stream_out_fmts[]?

> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[0],

nit: If you only need to point to these data once, you could define them in
place, as a compound literal, like

>                 .frmsizes = &(struct v4l2_framesizeenum) {
>                         // initialize here
>                 },

> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[1],
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,

link_flags seems to be 0 for all nodes.

> +		.capture = true,

We already know this from .buf_type. We can check using the
V4L2_TYPE_IS_OUTPUT() macro.

> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),

I don't think this is correct. The description suggests one specific format
(local motion vector histogram), so the queue should probably be hardwired
to that format?

> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,

Where does this number come from?

> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)

< ARRAY_SIZE(output_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)

< ARRAY_SIZE(capture_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;

Do we need this additional mtk_cam_dev struct? Couldn't we just use
mtk_isp_p1 here?

> +	mtk_cam_dev_queue_setup(cam_dev);
> +	/* v4l2 sub-device registration */
> +
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);

This debugging message doesn't seem very useful. Please remove.

> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret) {
> +		mtk_cam_v4l2_unregister(cam_dev);

Please use an error path on the bottom of the function instead. With goto
labels named after the next clean-up step that needs to be performed.

> +		return ret;
> +	}
> +
> +	spin_lock_init(&cam_dev->req_lock);
> +	INIT_LIST_HEAD(&cam_dev->req_list);
> +	mutex_init(&cam_dev->lock);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,

"release" is normally used for file_operations. How about "cleanup"?

> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	mutex_destroy(&cam_dev->lock);
> +
> +	return 0;
> +}

I'd suggest moving any generic API implementation code (platform_device,
V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
mtk_cam_hw.c.

> +
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"

Maybe it's not a critical thing, but generally it's a good practice to just
explicitly specific this name somewhere, e.g. struct
platform_driver::driver::name and then just refer to dev_driver_name(). It
makes it easier to make sure that the name stays the same everywhere.

> +
> +#define MTK_CAM_P1_META_IN_0			0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1

Since these are just some utility definitions, we can use enum instead of
assigning the values by hand.

> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> +#define MTK_CAM_P1_META_OUT_0			3
> +#define MTK_CAM_P1_META_OUT_1			4
> +#define MTK_CAM_P1_META_OUT_2			5
> +#define MTK_CAM_P1_META_OUT_3			6
> +#define MTK_CAM_P1_TOTAL_CAPTURE		6

Ditto.

> +
> +#define MTK_CAM_P1_TOTAL_NODES			7

Please just add the two totals together rather than manually specifying the
value.

> +
> +struct mtk_cam_dev_request {
> +	struct media_request	req;
> +	struct list_head	list;
> +};
> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +	unsigned int		node_id;
> +};

Could you add kerneldoc comments for the 2 structs?

> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + * @frmsizes:	supported frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device

vdev_pad?

> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *

Please fix the order of the documentation to match the order of the struct.

> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct video_device vdev;

Not documented above.

> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	/* protect for pending_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t slock;

How about calling it pending_list_lock?

> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @lock:       The mutex protecting video device open/release operations
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> + * @req_list:	Lins to keep media requests before streaming on
> + * @req_lock:	Protect the req_list data
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->

This probably isn't the best place for graph topology description. I think
we actually want a separate documentation file for this, similar to
Documentation/media/v4l-drivers/ipu3.rst.

> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	/* protect video device open/release operations */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	unsigned int streaming:1;
> +	atomic_t streamed_node_count;
> +	struct list_head req_list;
> +	/* protect for req_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t req_lock;

How about calling it req_list_lock?

Best regards,
Tomasz


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-10  9:54       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:54 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> Implement standard V4L2 video driver that utilizes V4L2
> and media framework APIs. In this driver, supports one media
> device, one sub-device and seven video devices during
> initialization. Moreover, it also connects with sensor and
> seninf drivers with V4L2 async APIs.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "media: support Mediatek sensor interface driver"[1].
> 
> ISP P1 sub-device communicates with seninf sub-device with CIO.
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/10979135/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
>  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
>  3 files changed, 1848 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> 

Thanks for the patch. Please see my comments inline.

[snip]

> +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> +				      struct media_request *new_req)
> +{
> +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> +
> +	if (!cam_dev->streaming) {
> +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> +		spin_lock(&cam_dev->req_lock);
> +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> +		spin_unlock(&cam_dev->req_lock);
> +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> +		return;
> +	}
> +
> +	/* Normal enqueue flow */
> +	if (new_req) {
> +		mtk_isp_req_enqueue(dev, new_req);
> +		return;
> +	}
> +
> +	/* Flush all media requests wehen first stream on */
> +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> +		list_del(&req->list);
> +		mtk_isp_req_enqueue(dev, &req->req);
> +	}
> +}

This will have to be redone, as per the other suggestions, but generally one
would have a function that tries to queue as much as possible from a list to
the hardware and another function that adds a request to the list and calls
the first function.

> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> +
> +	vb2_request_queue(req);
> +	mtk_cam_req_try_isp_queue(cam_dev, req);

Looks like this driver is suffering from versy similar problems in request
handling as the DIP driver used to.

I'd prefer to save my time and avoid repeating the same comments, so please
check my comments for the DIP driver and apply them to this one too:

https://patchwork.kernel.org/patch/10905223/

> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)

Doesn't this function return bits not bytes?

> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> +					__u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);

The __ prefixed types should be used only inside UAPI. Please change the
driver to use the normal ones.

> +
> +	width = ALIGN(width, 4);

If there is some alignment requirement for width, it should be handled by
TRY_/S_FMT and here we should already assume everything properly aligned.

> +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> +
> +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> +				       __u32 pix_fmt)
> +{
> +	__u32 stride;
> +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> +
> +	width = ALIGN(width, 4);

Ditto.

> +	stride = DIV_ROUND_UP(width * 3, 2);

Could we introduce a local variable for this intermediate value, so that its
name could explain what the value is?

> +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> +
> +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> +		stride = ALIGN(stride, 4);

Is it expected that only the F10 format needs this alignment?

> +
> +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> +
> +	return stride;
> +}
> +
> +static __u32 img_cal_stride(struct device *dev,
> +			    int node_id,
> +			    __u32 width,
> +			    __u32 pix_fmt)
> +{
> +	__u32 bpl;
> +
> +	/* Currently, only support one_pixel_mode */
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> +
> +	/* For DIP HW constrained, it needs 4 byte alignment */
> +	bpl = ALIGN(bpl, 4);
> +
> +	return bpl;
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	unsigned int i;
> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Calcuate mplane pix format */
> +static void
> +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> +			   struct v4l2_pix_format_mplane *dest_fmt,
> +			   unsigned int node_id)
> +{
> +	unsigned int i;
> +	__u32 bpl, sizeimage, imagsize;

Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?

> +
> +	imagsize = 0;
> +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> +		bpl = img_cal_stride(dev,
> +				     node_id,
> +				     dest_fmt->width,
> +				     dest_fmt->pixelformat);
> +		sizeimage = bpl * dest_fmt->height;
> +		imagsize += sizeimage;
> +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> +		memset(dest_fmt->plane_fmt[i].reserved,
> +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));

This memset is not needed. The core clears the reserved fields
automatically:

https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559

(We may want to backport the patch that added that to our 4.19 branch.)

> +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> +	}
> +
> +	if (dest_fmt->num_planes == 1)
> +		dest_fmt->plane_fmt[0].sizeimage = imagsize;

Hmm, we only seem to support 1 plane raw formats in this driver. Does the
hardware support any formats with more than 1 plane? If not, all the code
should be simplified to just assume 1 plane.

> +}
> +
> +static void
> +mtk_cam_dev_set_img_fmt(struct device *dev,
> +			struct v4l2_pix_format_mplane *dest_fmt,
> +			const struct v4l2_pix_format_mplane *src_fmt,
> +			unsigned int node_id)
> +{
> +	dest_fmt->width = src_fmt->width;
> +	dest_fmt->height = src_fmt->height;
> +	dest_fmt->pixelformat = src_fmt->pixelformat;
> +	dest_fmt->field = src_fmt->field;
> +	dest_fmt->colorspace = src_fmt->colorspace;
> +	dest_fmt->num_planes = src_fmt->num_planes;
> +	/* Use default */
> +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest_fmt->xfer_func =
> +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));

Given that src_fmt should already be validated and have any fields adjusted
to match the driver requirements, wouldn't all the lines above be equivalent
to *dest_fmt = *src_fmt?

We probably want to move setting all the constant fields to
mtk_cam_vidioc_try_fmt().

> +
> +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> +		__func__,
> +		(dest_fmt->pixelformat & 0xFF),
> +		(dest_fmt->pixelformat >> 8) & 0xFF,
> +		(dest_fmt->pixelformat >> 16) & 0xFF,
> +		(dest_fmt->pixelformat >> 24) & 0xFF,
> +		dest_fmt->width,
> +		dest_fmt->height);
> +
> +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);

This should have been called already before this function was called,
because src_fmt should be already expected to contain valid settings. In
fact, this is already called in mtk_cam_vidioc_try_fmt().

> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct device *dev,

Please don't pass struct device pointer around, but instead just the main
driver data struct, which should be much more convenient for accessing
various driver data. Please fix the other functions as well.

> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (queue_desc->image) {
> +		mtk_cam_dev_set_img_fmt(dev,
> +					&dest->fmt.pix_mp,
> +					&default_fmt->fmt.pix_mp,
> +					queue_desc->id);

We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
v4l2_format struct and have any incorrect fields replaced by
mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
v4l2_format at runtime.

> +	} else {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +	}
> +}
> +
> +static int mtk_cam_isp_open(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	int ret;
> +
> +	mutex_lock(&cam_dev->lock);
> +	ret = v4l2_fh_open(file);
> +	if (ret)
> +		goto unlock;
> +
> +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);

Please don't power on open. Normally applications keep the device nodes open
all the time, so they would keep everything powered on.

Normally this should be done as late as possible, ideally when starting the
streaming.

> +	if (ret)
> +		dev_err(dev, "%s fail:%d", __func__, ret);
> +
> +unlock:
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_isp_release(struct file *file)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	mutex_lock(&cam_dev->lock);
> +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> +	vb2_fop_release(file);
> +	mutex_unlock(&cam_dev->lock);
> +
> +	return 0;
> +}

If we remove power handling from open and release, we should be able to just
use v4l2_fh_open() and vb2_fop_release() directly in the
v4l2_file_operations struct.

> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> +{
> +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> +	struct media_entity *entity;
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct v4l2_subdev *sensor;

This variable would be unitialized if there is no streaming sensor. Was
there no compiler warning generated for this?

> +
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x\n",
> +			entity->name, entity->function);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "Sensor is not connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	/* Align vb2_core_streamon design */
> +	if (cam_dev->streaming) {
> +		dev_warn(dev, "already streaming\n", dev);
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	if (!cam_dev->seninf) {
> +		dev_err(dev, "no seninf connected:%d\n", ret);
> +		return -EPERM;

I don't think -EPERM is a good error code here. It's about a missing seninf
device, so perhaps -ENODEV?

> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> +	if (!cam_dev->sensor)
> +		return -EPERM;

> -ENODEV

> +
> +	ret = mtk_isp_config(dev);
> +	if (ret)
> +		return -EPERM;

Maybe just return ret?

> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "%s stream on failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		goto fail_sensor_on;
> +	}
> +
> +	cam_dev->streaming = true;
> +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> +	isp_composer_stream(dev, 1);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_on:
> +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +
> +	return -EPERM;

return ret?

> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	if (!cam_dev->streaming) {
> +		dev_warn(dev, "already stream off");
> +		return 0;
> +	}

Could we check this in the caller?

> +
> +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "%s stream off failed:%d\n",
> +			cam_dev->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	isp_composer_stream(dev, 0);

Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
can't guarantee that the hardware releases all the memory that we're going
to free once this function returns.

> +	cam_dev->streaming = false;
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	if (enable)
> +		return mtk_cam_cio_stream_on(cam_dev);
> +	else
> +		return mtk_cam_cio_stream_off(cam_dev);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> +{
> +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> +
> +	return on ? mtk_isp_power_init(cam_dev) :
> +		    mtk_isp_power_release(&cam_dev->pdev->dev);

s_power is a historical thing and we shouldn't be implementing it. Instead,
we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
whenever we start and stop streaming respectively.

> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)

I assume this check is needed, because the pads with higher indexes are not
video nodes? If so, a comment would be helpful here.

> +		cam_dev->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = &mtk_cam_dev->pdev->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);

This can be folded into the declaration.

> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> +		__func__,
> +		node->id,
> +		buf->vbb.request_fd,
> +		buf->vbb.vb2_buf.index);
> +
> +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> +	if (vb->vb2_queue->uses_requests)
> +		return;

I'd suggest removing non-request support from this driver. Even if we end up
with a need to provide compatibility for non-request mode, then it should be
built on top of the requests mode, so that the driver itself doesn't have to
deal with two modes.

> +
> +	/* Added the buffer into the tracking list */
> +	spin_lock(&node->slock);
> +	list_add_tail(&buf->list, &node->pending_list);
> +	spin_unlock(&node->slock);
> +
> +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *smem_dev = cam_dev->smem_dev;
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> +	buf->scp_addr = 0;

Just a reminder that this will have to be reworked according to my comments
for the memory allocation patch.

> +
> +	/* scp address is only valid for meta input buffer */
> +	if (node->desc.smem_alloc)
> +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> +							      buf->daddr);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size)
> +		return -EINVAL;

For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
Otherwise we could get not enough or invalid data.

> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);

This shouldn't be called on OUTPUT buffers.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {

We should also verify that *num_planes == 1, as we don't support more
planes in this driver.

> +		if (sizes[0] < size)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct mtk_cam_dev_request *req, *req0;
> +	struct media_request_object *obj, *obj0;
> +	struct vb2_buffer *vb;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> +
> +	/* Return all buffers */
> +	spin_lock(&node->slock);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {

nit: One would normally call the second argument "prev", or "b_prev".

> +		vb = &b->vbb.vb2_buf;
> +		if (vb->state == VB2_BUF_STATE_ACTIVE)

We shouldn't need to check the buffer state.

> +			vb2_buffer_done(vb, state);
> +		list_del(&b->list);
> +	}
> +	spin_unlock(&node->slock);
> +
> +	spin_lock(&cam_dev->req_lock);
> +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {

nit: Ditto.

> +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {

Need to check if the object is a buffer.

> +			vb = container_of(obj, struct vb2_buffer, req_obj);
> +			if (vb->state == VB2_BUF_STATE_ACTIVE)

vb->state shouldn't be accessed directly from the drivers.

Generally, the need to check the state here would suggest that there is
something wrong with how the driver manages the requests. The list that is
being iterated here shouldn't contain any requests that have buffers that
aren't active. That will be achieved if my comments for the request handling
in the DIP driver are applied to this driver as well.

> +				vb2_buffer_done(vb, state);
> +		}
> +		list_del(&req->list);
> +	}
> +	spin_unlock(&cam_dev->req_lock);
> +
> +	if (node->vbq.uses_requests)
> +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> +	int ret;
> +
> +	if (!node->enabled) {

How is this synchronized with mtk_cam_media_link_setup()?

> +		dev_err(dev, "Node:%d is not enable\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_no_link;
> +	}
> +
> +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> +		atomic_read(&cam_dev->streamed_node_count), node_count);
> +
> +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> +		return 0;

How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
between calls to this function on different video nodes?

> +
> +	/* Start streaming of the whole pipeline now */
> +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> +	if (ret) {
> +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> +		goto fail_start_pipeline;
> +	}
> +

Related to the above comment: If we start the media pipeline when we start
streaming on the first node, we would naturally prevent the link
configuration changes until the last node stops streaming (as long as the
link is not DYNAMIC). Note that it would only mark the entities as
streaming, but it wouldn't call their s_stream, which I believe is exactly
what we would need to solve the problem above.

> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> +		goto fail_stream_on;
> +	}
> +
> +	return 0;
> +
> +fail_stream_on:
> +	media_pipeline_stop(&node->vdev.entity);
> +fail_start_pipeline:
> +	atomic_dec(&cam_dev->streamed_node_count);
> +fail_no_link:
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = &cam_dev->pdev->dev;
> +
> +	if (!node->enabled)
> +		return;

It shouldn't be possible for this to happen, because nobody could have
called start_streaming on a disabled node.

> +
> +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);

Shouldn't we stop streaming first, so that the hardware operation is
cancelled and any buffers owned by the hardware are released?

> +
> +	dev_dbg(dev, "%s: count info:%d", __func__,
> +		cam_dev->subdev.entity.stream_count);
> +
> +	/* Check the first node to stream-off */
> +	if (!cam_dev->subdev.entity.stream_count)
> +		return;
> +
> +	media_pipeline_stop(&node->vdev.entity);
> +
> +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> +		dev_err(dev, "failed to stop streaming\n");
> +}
> +
> +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	v4l2_ctrl_request_complete(vb->req_obj.req,
> +				   dev->v4l2_dev.ctrl_handler);

This would end up being called multiple times, once for each video node.
Instead, this should be called explicitly by the driver when it completed
the request - perhaps in the frame completion handler?

With that, we probably wouldn't even need this callback.

> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +
> +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));

We could just use dev_driver_name(cam_dev->dev) for both.

> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam_dev->media_dev.dev));

We should just store dev in cam_dev.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;

Is the set of formats available always the same regardless of the sensor
format?

> +	f->flags = 0;

We need f->description too.

> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (!node->desc.num_fmts)
> +		return -EINVAL;

When would that condition happen?

> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *in_fmt)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	const struct v4l2_format *dev_fmt;
> +	__u32  width, height;

Don't use __ types in implementation, they are here for UAPI purposes. There
is u32, which you could use instead, but for width and height you don't need
explicit size, so unsigned int should be good.

> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> +		__func__,
> +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> +
> +	width = in_fmt->fmt.pix_mp.width;
> +	height = in_fmt->fmt.pix_mp.height;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> +				       in_fmt->fmt.pix_mp.pixelformat);
> +	if (dev_fmt) {
> +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +					&in_fmt->fmt.pix_mp,
> +					&dev_fmt->fmt.pix_mp,
> +					node->id);
> +	} else {
> +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +					     &node->desc, in_fmt);

We shouldn't just load a default format. This function should validate all
the fields one by one and adjust them to something appropriate.

> +	}

CodingStyle: No braces if both if and else bodies have only 1 statement
each.

> +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> +					   width,
> +					   CAM_MIN_WIDTH,
> +					   in_fmt->fmt.pix_mp.width);

Shouldn't we clamp this with some maximum value too?

> +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> +					    height,
> +					    CAM_MIN_HEIGHT,
> +					    in_fmt->fmt.pix_mp.height);

Ditto.

> +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> +				   &in_fmt->fmt.pix_mp, node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (cam_dev->streaming)
> +		return -EBUSY;

I think this should rather be something like vb2_queue_is_busy(), which
would prevent format changes if buffers are allocated.

> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +
> +	/* Configure to video device */
> +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> +				&node->vdev_fmt.fmt.pix_mp,
> +				&f->fmt.pix_mp,
> +				node->id);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> +				     struct v4l2_input *input)
> +{
> +	if (input->index)
> +		return -EINVAL;
> +
> +	strscpy(input->name, "camera", sizeof(input->name));
> +	input->type = V4L2_INPUT_TYPE_CAMERA;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> +				  unsigned int *input)
> +{
> +	*input = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_input(struct file *file,
> +				  void *fh, unsigned int input)
> +{
> +	return input == 0 ? 0 : -EINVAL;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	strscpy(f->description, node->desc.description,
> +		sizeof(node->desc.description));
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +	.s_power = mtk_cam_sd_s_power,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_ops = {

nit: mtk_cam_media_entity_ops?

> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = mtk_cam_isp_open,
> +	.release = mtk_cam_isp_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_req_ops = {

nit: Those are media ops, so perhaps just mtk_cam_media_ops?

> +	.link_notify = v4l2_pipeline_link_notify,
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct device *dev,
> +				  struct media_device *media_dev)
> +{
> +	media_dev->dev = dev;
> +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,

Could we replace any use of this macro with dev_driver_string(dev) and then
delete the macro? The less name strings in the driver the better, as there
is less change for confusing the userspace if few different names are used
at the same time.

> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_req_ops;
> +
> +	return media_device_register(media_dev);
> +}
> +
> +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];

Would it make sense to pass node as an argument to this function instead of
(or in addition to) i?

> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;

Why not call it capture instead and avoid the inversion?

> +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> +	int ret;
> +
> +	cam_dev->subdev_pads[i].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->enabled = false;

Are all the nodes optional? If there is any required node, it should be
always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.

> +	node->id = i;
> +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;

Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?

> +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> +				     &node->desc,
> +				     &node->vdev_fmt);
> +
> +	/* Initialize vbq */
> +	vbq->type = node->vdev_fmt.type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam_dev->smem_dev;
> +	} else {
> +		vbq->dev = &cam_dev->pdev->dev;
> +	}
> +
> +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		vdev->entity.function =
> +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;

This is a video node, so it's just a DMA, not a processing entity. I believe
all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.

> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> +	/* Put the process hub sub device in the vb2 private data */

What is "process hub" and what "sub device" is this about?

> +	vbq->drv_priv = cam_dev;
> +	vbq->lock = &node->lock;
> +	vbq->supports_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_vb2_queue;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->lock;
> +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.ops = NULL;
> +	/* Enable private control for image video devices */
> +	if (node->desc.image) {
> +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> +		vdev->ctrl_handler = &node->ctrl_handler;
> +	}
> +	video_set_drvdata(vdev, cam_dev);
> +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vdev;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam_dev->subdev.entity,
> +					    i, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> +					    i, &vdev->entity, 0,
> +					    link_flags);
> +	}
> +	if (ret)
> +		goto fail_link;
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->lock);
> +	spin_lock_init(&node->slock);
> +	INIT_LIST_HEAD(&node->pending_list);

This should be all initialized before registering the video device.
Otherwise userspace could open the device before these are initialized.

> +
> +	return 0;
> +
> +fail_link:
> +	video_unregister_device(vdev);
> +fail_vdev:
> +	vb2_queue_release(vbq);
> +fail_vb2_queue:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)

This function doesn't have anything to do with mem2mem. How about
mtk_cam_v4l2_register()?

Perhaps it would make sense to move any media related code into into
mtk_cam_media_register(), keep only V4L2 related code here and have the
caller call the former first and then this one, rather than having such deep
sequence of nested calls, which makes the driver harder to read.

> +{
> +	struct device *dev = &cam_dev->pdev->dev;

How about just storing dev, instead of pdev in the struct? Also, calling the
argument "cam", would make it as short as cam->dev.

> +	/* Total pad numbers is video devices + one seninf pad */
> +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = mtk_cam_media_register(dev,
> +				     &cam_dev->media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +	dev_info(dev, "Register media device: %s, 0x%pK",
> +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);

An info message should be useful to the user in some way. Printing kernel
pointers isn't useful. Something like "registered media0" could be useful to
let the user know which media device is associated with this driver if there
is more than one in the system.

> +
> +	/* Set up v4l2 device */
> +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_v4l2_dev;

Please call the labels after the cleanup step that needs to be done. It
makes it easier to spot any ordering errors.

> +	}
> +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);

Same as above.

> +
> +	/* Initialize subdev media entity */
> +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> +					    sizeof(*cam_dev->subdev_pads),
> +					    GFP_KERNEL);
> +	if (!cam_dev->subdev_pads) {
> +		ret = -ENOMEM;
> +		goto fail_subdev_pads;
> +	}
> +
> +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> +				     num_subdev_pads,
> +				     cam_dev->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed initialize media pads:%d:\n", ret);

Stray ":" at the end of the message.

> +		goto fail_subdev_pads;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_subdev_pads; i++)
> +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> +		 "%s", MTK_CAM_DEV_P1_NAME);
> +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> +
> +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> +		goto fail_subdev;
> +	}
> +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		ret = mtk_cam_video_register_device(cam_dev, i);
> +		if (ret)
> +			goto fail_video_register;
> +	}
> +
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_video_register:
> +	i--;

This could be moved into the for clause, as the initialization statement.

> +	for (; i >= 0; i--) {

i is unsigned. Did this compile without warnings?

> +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);

Should we move this into mtk_cam_video_unregister_device() to be consistent
with registration?

> +	}
> +fail_subdev:
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +fail_subdev_pads:
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +fail_v4l2_dev:
> +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);

Please print errors only where they actually happen, not at the cleanup.

> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i;
> +	struct mtk_cam_video_device *dev;

nit: Move the declaration inside the for loop, since the variable is only
used there.

> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		dev = &cam_dev->vdev_nodes[i];
> +		video_unregister_device(&dev->vdev);
> +		media_entity_cleanup(&dev->vdev.entity);
> +		mutex_destroy(&dev->lock);
> +		if (dev->desc.image)
> +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> +	}
> +
> +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> +
> +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> +	media_entity_cleanup(&cam_dev->subdev.entity);
> +	kfree(cam_dev->subdev_pads);

This was allocated using devm_kcalloc(), so no need to free it explicitly.

> +
> +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> +	media_device_unregister(&cam_dev->media_dev);
> +	media_device_cleanup(&cam_dev->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> +				    MTK_CAM_CIO_PAD_SRC,
> +				    &cam_dev->subdev.entity,
> +				    MTK_CAM_CIO_PAD_SINK,
> +				    0);
> +	if (ret) {
> +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> +			cam_dev->seninf->entity.name,
> +			cam_dev->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	dev_info(dev, "Complete the v4l2 registration\n");

dev_dbg()

> +
> +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}

Why not just put the contents of this function inside 
mtk_cam_dev_notifier_complete()?

> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +

Should we somehow check that the entity we got is seninf indeed and there
was no mistake in DT?

> +	cam_dev->seninf = sd;
> +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);

bound

Also please make this dev_dbg().

> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam_dev =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam_dev->seninf = NULL;
> +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);

unbound

> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	return mtk_cam_dev_complete(notifier);
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	int ret;
> +
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> +		NULL);
> +	if (ret)
> +		return ret;
> +
> +	if (!cam_dev->notifier.num_subdevs)
> +		return -ENODEV;

Could we print some error messages for the 2 cases above?

> +
> +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");

dev_dbg()

> +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> +					   &cam_dev->notifier);
> +	if (ret) {
> +		dev_err(&cam_dev->pdev->dev,
> +			"failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> +{
> +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> +	.vidioc_s_input = mtk_cam_vidioc_s_input,

I don't think we need vidioc_*_input. At least the current implementation in
this patch doesn't seem to do anything useful.

> +	/* buffer queue management */

Drop this comment, as it's obvious that the callbacks with "buf" in the name
are related to buffers, there are some non-buffer callbacks below too (e.g.
vidioc_subscribe_event) and also the other ops structs below don't have such
comment.

> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 128 * PAGE_SIZE,

PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
making it 512 * SZ_1K?

That said, it should normally be just sizeof(struct some_struct_used_here).

Same for the other entries below.

> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 300 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 160 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 72 * PAGE_SIZE,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_SRGB,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = RRZ_MAX_WIDTH,
> +			.height = RRZ_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> +			.field = V4L2_FIELD_NONE,
> +			.colorspace = V4L2_COLORSPACE_RAW,
> +			.num_planes = 1,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = IMG_MAX_WIDTH,
> +			.min_width = IMG_MIN_WIDTH,
> +			.max_height = IMG_MAX_HEIGHT,
> +			.min_height = IMG_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +	{
> +		.index = 0,
> +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +		.stepwise = {
> +			.max_width = RRZ_MAX_WIDTH,
> +			.min_width = RRZ_MIN_WIDTH,
> +			.max_height = RRZ_MAX_HEIGHT,
> +			.min_height = RRZ_MIN_HEIGHT,
> +			.step_height = 1,
> +			.step_width = 1,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.description = "ISP tuning parameters",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.capture = false,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 1,

Why not just make it always 0 and move the default format to the beginning
of stream_out_fmts[]?

> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[0],

nit: If you only need to point to these data once, you could define them in
place, as a compound literal, like

>                 .frmsizes = &(struct v4l2_framesizeenum) {
>                         // initialize here
>                 },

> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 1,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &img_frm_size_nums[1],
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.description = "AE/AWB histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.description = "AF histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.description = "Local contrast enhanced statistics",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.capture = true,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.description = "Local motion vector histogram",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,

link_flags seems to be 0 for all nodes.

> +		.capture = true,

We already know this from .buf_type. We can check using the
V4L2_TYPE_IS_OUTPUT() macro.

> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.num_fmts = ARRAY_SIZE(meta_fmts),

I don't think this is correct. The description suggests one specific format
(local motion vector histogram), so the queue should probably be hardwired
to that format?

> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,

Where does this number come from?

> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int i, node_idx;
> +
> +	node_idx = 0;
> +
> +	/* Setup the output queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)

< ARRAY_SIZE(output_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)

< ARRAY_SIZE(capture_queues)

> +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev)
> +{
> +	int ret;
> +
> +	cam_dev->pdev = pdev;

Do we need this additional mtk_cam_dev struct? Couldn't we just use
mtk_isp_p1 here?

> +	mtk_cam_dev_queue_setup(cam_dev);
> +	/* v4l2 sub-device registration */
> +
> +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> +		MTK_CAM_DEV_P1_NAME);

This debugging message doesn't seem very useful. Please remove.

> +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam_dev);
> +	if (ret) {
> +		mtk_cam_v4l2_unregister(cam_dev);

Please use an error path on the bottom of the function instead. With goto
labels named after the next clean-up step that needs to be performed.

> +		return ret;
> +	}
> +
> +	spin_lock_init(&cam_dev->req_lock);
> +	INIT_LIST_HEAD(&cam_dev->req_list);
> +	mutex_init(&cam_dev->lock);
> +
> +	return 0;
> +}
> +
> +int mtk_cam_dev_release(struct platform_device *pdev,

"release" is normally used for file_operations. How about "cleanup"?

> +			struct mtk_cam_dev *cam_dev)
> +{
> +	mtk_cam_v4l2_async_unregister(cam_dev);
> +	mtk_cam_v4l2_unregister(cam_dev);
> +
> +	mutex_destroy(&cam_dev->lock);
> +
> +	return 0;
> +}

I'd suggest moving any generic API implementation code (platform_device,
V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
mtk_cam_hw.c.

> +
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
new file mode 100644
index 000000000000..825cdf20643a
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
@@ -0,0 +1,173 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_DEV_V4L2_H__
> +#define __MTK_CAM_DEV_V4L2_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"

Maybe it's not a critical thing, but generally it's a good practice to just
explicitly specific this name somewhere, e.g. struct
platform_driver::driver::name and then just refer to dev_driver_name(). It
makes it easier to make sure that the name stays the same everywhere.

> +
> +#define MTK_CAM_P1_META_IN_0			0
> +#define MTK_CAM_P1_TOTAL_OUTPUT		1

Since these are just some utility definitions, we can use enum instead of
assigning the values by hand.

> +
> +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> +#define MTK_CAM_P1_META_OUT_0			3
> +#define MTK_CAM_P1_META_OUT_1			4
> +#define MTK_CAM_P1_META_OUT_2			5
> +#define MTK_CAM_P1_META_OUT_3			6
> +#define MTK_CAM_P1_TOTAL_CAPTURE		6

Ditto.

> +
> +#define MTK_CAM_P1_TOTAL_NODES			7

Please just add the two totals together rather than manually specifying the
value.

> +
> +struct mtk_cam_dev_request {
> +	struct media_request	req;
> +	struct list_head	list;
> +};
> +
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer	vbb;
> +	struct list_head	list;
> +	/* Intenal part */
> +	dma_addr_t		daddr;
> +	dma_addr_t		scp_addr;
> +	unsigned int		node_id;
> +};

Could you add kerneldoc comments for the 2 structs?

> +
> +/*
> + * struct mtk_cam_dev_node_desc - node attributes
> + *
> + * @id:		 id of the context queue
> + * @name:	 media entity name
> + * @description: descritpion of node
> + * @cap:	 mapped to V4L2 capabilities
> + * @buf_type:	 mapped to V4L2 buffer type
> + * @dma_port:	 the dma port associated to the buffer
> + * @link_flags:	 default media link flags
> + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> + * @capture:	 true for capture queue (device to user)
> + *		 false for output queue (from user to device)
> + * @image:	 true for image node, false for meta node
> + * @num_fmts:	 the number of supported formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum V4L2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts:	supported format
> + * @frmsizes:	supported frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	char *name;
> +	char *description;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 capture:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure.
> + *
> + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> + * @enabled:	Indicate the device is enabled or not
> + * @vdev_fmt:	The V4L2 format of video device
> + * @vdev_apd:	The media pad graph object of video device

vdev_pad?

> + * @vbq:	A videobuf queue of video device
> + * @desc:	The node attributes of video device
> + * @ctrl_handler:	The control handler of video device
> + * @pending_list:	List for pending buffers before enqueuing into driver
> + * @lock:	Serializes vb2 queue and video device operations.
> + * @slock:	Protect for pending_list.
> + *

Please fix the order of the documentation to match the order of the struct.

> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct v4l2_format vdev_fmt;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct video_device vdev;

Not documented above.

> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct list_head pending_list;
> +	/* Used for vbq & vdev */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	/* protect for pending_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t slock;

How about calling it pending_list_lock?

> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @pdev:	Pointer to platform device
> + * @smem_pdev:	Pointer to shared memory platform device
> + * @pipeline:	Media pipeline information
> + * @media_dev:	Media device
> + * @subdev:	The V4L2 sub-device
> + * @v4l2_dev:	The V4L2 device driver
> + * @notifier:	The v4l2_device notifier data
> + * @subdev_pads: Pointer to the number of media pads of this sub-device
> + * @ctrl_handler: The control handler
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> + * @seninf:	Pointer to the seninf sub-device
> + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> + * @lock:       The mutex protecting video device open/release operations
> + * @streaming:	Indicate the overall streaming status is on or off
> + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> + * @req_list:	Lins to keep media requests before streaming on
> + * @req_lock:	Protect the req_list data
> + *
> + * Below is the graph topology for Camera IO connection.
> + * sensor 1 (main) --> sensor IF --> P1 sub-device
> + * sensor 2 (sub)  -->

This probably isn't the best place for graph topology description. I think
we actually want a separate documentation file for this, similar to
Documentation/media/v4l-drivers/ipu3.rst.

> + *
> + */
> +struct mtk_cam_dev {
> +	struct platform_device *pdev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	/* protect video device open/release operations */

It's already documented in the kerneldoc comment.

> +	struct mutex lock;
> +	unsigned int streaming:1;
> +	atomic_t streamed_node_count;
> +	struct list_head req_list;
> +	/* protect for req_list */

It's already documented in the kerneldoc comment.

> +	spinlock_t req_lock;

How about calling it req_list_lock?

Best regards,
Tomasz


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-06-11  3:53     ` Jungo Lin
  (?)
@ 2019-07-10  9:56       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
>  4 files changed, 1457 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..9e59a6bfc6b7
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,126 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT			BIT(0)
> +#define CMOS_EN_BIT			BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST			BIT(0)
> +#define HW_PASS1_DON_ST		BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define SW_PASS1_DON_ST		BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST			BIT(4)
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +#define PSO_DONE_ST			BIT(14)
> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM		( \
> +					VS_INT_ST |\
> +					SOF_INT_ST |\
> +					HW_PASS1_DON_ST |\
> +					SW_PASS1_DON_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM		( \
> +					SOF_INT_ST |\
> +					SW_PASS1_DON_ST |\
> +					HW_PASS1_DON_ST |\
> +					VS_INT_ST |\
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					RRZO_ERR_ST |\
> +					AFO_ERR_ST |\
> +					IMGO_ERR_ST |\
> +					AAO_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM		( \
> +					AAO_DONE_ST |\
> +					AFO_DONE_ST)

Could we define the values next to the addresses of registers they apply to?
Also without the _BIT suffix and with the values prefixed with register
names. For example:

#define REG_TG_SEN_MODE		        0x0230
#define TG_SEN_MODE_CMOS_EN		BIT(0)

#define REG_TG_VF_CON			0x0234
#define TG_VF_CON_VFDATA_EN		BIT(0)

> +
> +/* Status check */
> +#define REG_CTL_EN			0x0004
> +#define REG_CTL_DMA_EN			0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2			0x0018
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_VF_CON			0x0234
> +
> +#define REG_IMGO_BASE_ADDR		0x1020
> +#define REG_RRZO_BASE_ADDR		0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT		0x1360
> +#define REG_RRZO_ERR_STAT		0x1364
> +#define REG_AAO_ERR_STAT		0x1368
> +#define REG_AFO_ERR_STAT		0x136c
> +#define REG_LCSO_ERR_STAT		0x1370
> +#define REG_UFEO_ERR_STAT		0x1374
> +#define REG_PDO_ERR_STAT		0x1378
> +#define REG_BPCI_ERR_STAT		0x137c
> +#define REG_LSCI_ERR_STAT		0x1384
> +#define REG_PDI_ERR_STAT		0x138c
> +#define REG_LMVO_ERR_STAT		0x1390
> +#define REG_FLKO_ERR_STAT		0x1394
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +#define REG_HW_FRAME_NUM		0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX		0x14dc
> +#define REG_META1_VB2_INDEX		0x151c

I don't believe these registers are really for VB2 indexes.

> +
> +/* FBC */
> +#define REG_AAO_FBC_STATUS		0x013c
> +#define REG_AFO_FBC_STATUS		0x0134
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..c5a3babed69d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +#include <media/v4l2-event.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);

Please move below. Just above the platform_driver struct where it's used.

> +
> +/* List of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +};

Please move inside mtk_isp_probe, as a static const local variable. That
could also let you shorten the name, to clk_names for example.

> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +					 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;

nit: You can just initialize the structure in the declaration.

> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +}
> +
> +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> +				   unsigned int request_fd,
> +				   unsigned int frame_seq_no,
> +				   struct list_head *list_buf,
> +				   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	u64    timestamp;

Too many spaces between u64 and timestamp.

> +
> +	if (!cam_dev->streaming)
> +		return;
> +
> +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> +		__func__, request_fd, frame_seq_no, state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	timestamp = ktime_get_ns();
> +	list_for_each_entry_safe(buf, b0, list_buf, list) {

nit: s/b0/buf_prev/

> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = timestamp;
> +		buf->vbb.sequence = frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

Any buffer that is not active shouldn't be on this list. If it is then it's
a bug somewhere else in the driver. Could be possibly related to the request
handling issues I pointed out in another comment.

> +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,

dequeue

> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct vb2_buffer *vb;
> +
> +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> +		return;
> +
> +	spin_lock(&node->slock);
> +	b = list_first_entry(&node->pending_list,
> +			     struct mtk_cam_dev_buffer,
> +			     list);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		vb = &b->vbb.vb2_buf;
> +		if (!vb->vb2_queue->uses_requests &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vb->timestamp = ktime_get_ns();
> +			b->vbb.sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +			list_del(&b->list);
> +			break;
> +		}
> +	}
> +	spin_unlock(&node->slock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,

dequeue

> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_warn(dev,
> +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> +				 framejob->frame_seq_no,
> +				 atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		} else {
> +			break;
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)

dequeue

> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> +		    event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int addr_offset;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	isp_dev->sof_count += 1;
> +
> +	if (cq_num <= frame_num) {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> +		return cq_num;
> +	}
> +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> +
> +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(isp_dev->dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> +		cq_num, frame_num, addr_offset);
> +
> +	return cq_num;
> +}
> +
> +static void irq_handle_notify_event(struct isp_device *isp_dev,
> +				    unsigned int irq_status,
> +				    unsigned int dma_status,
> +				    bool sof_only)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned long flags;
> +	int i;
> +
> +	if (irq_status & VS_INT_ST) {
> +		/* Notify specific HW events to user space */
> +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> +					     isp_dev->current_frame);

Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
definition of the V4L2_EVENT_FRAME_SYNC event at
https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html

> +		dev_dbg(dev,
> +			"frame sync is sent:%d:%d\n",
> +			isp_dev->sof_count,
> +			isp_dev->current_frame);
> +		if (sof_only)
> +			return;

If this function can be called only to perform this block, perhaps it should
be split into two functions?

Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
irq_status? Is it expected that in such case the other part of the function
is executed?

> +	}
> +
> +	if (irq_status & SW_PASS1_DON_ST) {
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask =
> +		(irq_status & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask =
> +		(dma_status & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);

I can see that all isp_deque_work() does is returning buffers to vb2. I
don't think we need this intricate system to do that, as we could just do
it inside the interrupt handler, in isp_irq_cam() directly.

> +
> +	dev_dbg(dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irq_status & INT_ST_MASK_CAM),
> +		(dma_status & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cam_idx, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned int aao_fbc, afo_fbc;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev.streaming)
> +		return IRQ_HANDLED;

This shouldn't be needed. The driver needs to unmask the interrupts in
hardware registers when it starts streaming and mask them when it stops.
Note that I mean the P1 hardware registers, not disable_irq(), which
shouldn't be used.

> +
> +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);

Hmm, reading vb2 buffer index from hardware registers? Was this hardware
designed exclusively for V4L2? ;)

Jokes aside, how does the hardware know V4L2 buffer indexes?

> +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> +		return IRQ_HANDLED;

Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
get an interrupt without any unmasked hardware IRQs active in the status,
that's an error somewhere and we should return IRQ_NONE.

> +
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +
> +	/* Sof, done order check */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue frame */
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> +		isp_dev->current_frame = hw_frame_num;

What exactly is hw_frame_num? Shouldn't we assign it before notifying the
event?

> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irq_status & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> +	}

The if and else blocks do almost the same things just in different order. Is
it really expected?

> +
> +	if (irq_status & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x\n",
> +			irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irq_status & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,

Please just put that string here, otherwise the reader would have no idea
what message is being printed here.

> +			'A' + cam_idx,
> +			isp_dev->sof_count,
> +			irq_status,
> +			dma_status,
> +			hw_frame_num,
> +			cq_num,
> +			aao_fbc,
> +			afo_fbc);

nit: No need to put each argument in its own line.

> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}

This look-up should be done once in probe(). Only the rproc_boot() should
happen dynamically.

> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +	mutex_init(&isp_ctx->lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +
> +	return 0;

Everything here looks like something that should be done once in probe. I
also don't see a point of having a separate mtk_isp_p1_ctx struct for the
data above. It could be just located in p1_dev, at least for now.

If we end up implementing support for multiple contexts, we could isolate
per-context data then, once we know what's really per-context. For now,
let's keep it simple.

> +}
> +
> +static int isp_uninit_context(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	if (isp_ctx->isp_deque_thread.thread) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	mutex_destroy(&isp_ctx->lock);
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int enabled_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enabled_dma_ports = 0;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		if (cam_dev->vdev_nodes[i].enabled)
> +			enabled_dma_ports |=
> +				cam_dev->vdev_nodes[i].desc.dma_port;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> +
> +	return enabled_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return RAW_PXL_ID_R;
> +	default:

Could we fail here instead?

> +		return RAW_PXL_ID_B;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return IMG_FMT_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return IMG_FMT_FG_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:

Could we fail here instead, so that we don't mask some potential errors?

> +		return 10;
> +	}
> +}
> +
> +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> +			   const struct v4l2_format *in_fmt,
> +			   const struct v4l2_subdev_format *sd_format)
> +{
> +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> +
> +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;

Please group operations on the same field together, i.e. remove the blank
line above size.stride and add one blank line above size.w.

> +
> +	out_fmt->crop.left = 0x0;
> +	out_fmt->crop.top = 0x0;
> +

Remove the blank line.

> +	out_fmt->crop.width = sd_format->format.width;
> +	out_fmt->crop.height = sd_format->format.height;
> +
> +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> +		  "img out:%d:%d in:%d:%d",
> +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> +		  out_fmt->crop.width, out_fmt->crop.height);

We should check this earlier and fail the streaming start if there is a
mismatch between sensor and video node configuration.

> +
> +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> +		out_fmt->pixel_byte,
> +		out_fmt->img_fmt);
> +	dev_dbg(dev,
> +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = isp_setup_scp_rproc(p1_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = isp_init_context(p1_dev);
> +	if (ret)
> +		return ret;

The above function doesn't really seem to be related to power management.
Should it be called from subdev stream on?

> +
> +	ret = isp_composer_init(dev);
> +	if (ret)
> +		goto composer_err;

This also doesn't seem to be related to power management.

> +
> +	pm_runtime_get_sync(dev);
> +
> +	/* ISP HW INIT */
> +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;

There is some amount of code handling various values of isp_hw_module in
this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
code that can't be tested. Please either add support for all the indexes in
the driver or simplify all the code to just handle CAM_B.

> +	/* Use pure RAW as default HW path */
> +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> +
> +	isp_composer_hw_init(dev);
> +	/* Check enabled DMAs which is configured by media setup */
> +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));

Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
necessary?

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +
> +composer_err:
> +	isp_uninit_context(dev);
> +
> +	return ret;
> +}
> +
> +int mtk_isp_power_release(struct device *dev)
> +{
> +	isp_composer_hw_deinit(dev);
> +	isp_uninit_context(dev);

These two don't seem to be related to power either.

Instead, I don't see anything that could undo the rproc_boot() operation
here.

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_fmt;
> +	unsigned int enabled_dma_ports;
> +	struct v4l2_format *img_fmt;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> +
> +	/* Sensor config */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> +

Unnecessary blank line.

> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	dev_dbg(dev,
> +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_fmt.format.width, sd_fmt.format.height,
> +		sd_fmt.format.code, sd_fmt.format.field,
> +		sd_fmt.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* Fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = 0x1;
> +	/* Support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;

Why hexadecimal numerals? Also, what's the meaning of these values? For
anything boolean, you could just use true and false as a substite of 0 and
1. For anything that has more values, please define the values using macros.

> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_fmt.format.code);
> +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;

Move the other crop settings from above to here.

> +
> +	config_param.cfg_main_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {

No need for the == R_IMGO part.

> +		config_param.cfg_main_param.bypass = 0;
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	config_param.cfg_resize_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {

Ditto.

> +		config_param.cfg_resize_param.bypass = 0;
> +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;

Should image DMAs be masked out of this bitmap?

> +
> +	isp_composer_hw_config(dev, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> +		     struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> +}
> +
> +void mtk_isp_req_flush_buffers(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_queue_job *job, *j0;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> +
> +	if (!atomic_read(&p1_list->queue_cnt))
> +		return;

Do we need this explicit check? The code below wouldn't do anything if there
isn't anything in the list. IMHO we could even remove queue_cnt completely.

> +
> +	spin_lock(&p1_list->lock);
> +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {

nit: s/j0/job_prev/

> +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {

nit: s/b0/buf_pref/

Also, we should be able to replace this with iterating over the generic list
of request objects, rather than this internal list.

> +			list_del(&buf->list);
> +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

It shouldn't be possible to happen. If you see this check failing, that
means a problem somewhere else in the driver.

> +				vb2_buffer_done(&buf->vbb.vb2_buf,
> +						VB2_BUF_STATE_ERROR);
> +		}
> +		list_del(&job->list_entry);
> +		atomic_dec(&p1_list->queue_cnt);
> +		kfree(job);
> +	}
> +	spin_unlock(&p1_list->lock);
> +}
> +
> +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct media_request_object *obj, *obj_safe;
> +	struct vb2_buffer *vb;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);

This allocation shouldn't be needed. The structure should be already a part
of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
even say that the contents of this struct should be just moved to that one
to avoid overabstracting.

> +	memset(framejob, 0, sizeof(*framejob));

Putting the above comment aside, kzalloc() already zeroes the memory.

> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;

How is this synchronized with the sof_count increment in irq_handle_sof()?

> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +		vb = container_of(obj, struct vb2_buffer, req_obj);

We should check that the object type before assuming it's a vb2_buffer by
calling vb2_request_object_is_buffer().

> +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +		framejob->request_fd = buf->vbb.request_fd;

We shouldn't use request_fd as the key here. We already have req here, which
is the right key to use.

That said, I can see framejob->request_fd only used for printing a debugging
message in mtk_cam_dev_job_finish(). The request API core should already
print something for us once a request is completed, so perhaps that isn't
needed?

> +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> +		list_add_tail(&buf->list, &framejob->list_buf);

Why do we need this private list? We could just call exactly the same
list_for_each() over the request objects.

> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);

We already have a list head in mtk_cam_dev_request.

> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		framejob->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s\n", __func__);

dev_dbg()

> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> +				      p1_dev->isp_ctx.clk_list);
> +	if (ret)
> +		goto clk_err;
> +
> +	return 0;
> +
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);

clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
this would disable the clocks second time.

> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);

dev_dbg()

> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);
> +}

There is no point in having wrapper functions to just call one function
inside. Please just call clk_bulk_*() directly.

> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check if the device isn't already runtime suspended. If it is, we
don't have to do anything here and can just return.


We also need to ensure that no new requests are queued to the hardware at
this point. This could be done by replacing any of the kthreads with
workqueues and making all of the workqueues freezable.

> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* Disable view finder */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, the TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}

Are you sure this is the right handling? We need to make sure the hardware
finishes processing the current frame and stops.

> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check runtime PM status here as well, because if the device was
suspended, there is nothing to do here.

> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev.streaming) {
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}

Does the hardware keep all the state, e.g. queued buffers, during suspend?
Would the code above continue all the capture from the next buffer, as
queued by the userspace before the suspend?

> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq;
> +	int ret;
> +	unsigned int i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;

Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
reference to platform_device is very useful later in the code.

> +
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {

I think we want to start from 0 here?

> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;

p1_dev already includes a pointer to this.

> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> +			i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)

devm_ioremap_resource() returns ERR_PTR() not NULL on error.

> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* Support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* Reg & interrupts index is shifted with 1  */

The reader can already see that it's shifted from the code below. The
comment should say _why_ it is shifted.

> +			irq = platform_get_irq(pdev, i - 1);

The bindings have 3 IRQs, but we only seem to request 2 here. Is that
expected?

> +			if (irq) {

Please invert this if, so that it bails out on error. Also, this should
check for negative errors codes, not 0.

> +				ret = devm_request_irq(dev, irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),

Use dev_name().

> +						       (void *)isp_dev);

No need to cast to void *.

> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						irq);
> +					return ret;
> +				}
> +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> +					irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}

We might want to move out the body of this loop to a separate function to
keep this function shorter.

> +
> +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_ctx.clk_list =

nit: "list" is the term commonly used for the list data structure. There is
also a convention to call the length of array XXX num_XXX, so how about
clks and num_clks?

> +		devm_kcalloc(dev,
> +			     p1_dev->isp_ctx.num_clks,
> +			     sizeof(*p1_dev->isp_ctx.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_ctx.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];

Shouldn't this be clk_list[i].id?

> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_ctx.num_clks,
> +				p1_dev->isp_ctx.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> +		goto err_init;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret)
> +		goto err_init;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +
> +err_init:
> +	if (p1_dev->cam_dev.smem_dev)
> +		device_unregister(p1_dev->cam_dev.smem_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)

For V4L2 drivers system and runtime PM ops would normally be completely
different. Runtime PM ops would be called when the hardware is idle already
or is about to become active. System PM ops would be called at system power
state change and the hardware might be both idle or active. Please also see
my comments to mtk_isp_suspend() and mtk_isp_resume() above.

> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",

Shouldn't this be something like mtk-cam-p1? Please make sure this
reasonably consistent with other drivers.

> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");

Mediatek Camera P1 ISP driver? Please make sure this is reasonably
consistent with other drivers (SenInf, DIP, FD).

> +MODULE_LICENSE("GPL");

GPL v2

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-scp.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define CAM_A_MAX_WIDTH		3328
> +#define CAM_A_MAX_HEIGHT		2496
> +#define CAM_B_MAX_WIDTH		5376
> +#define CAM_B_MAX_HEIGHT		4032
> +
> +#define CAM_MIN_WIDTH			80
> +#define CAM_MIN_HEIGHT			60
> +
> +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define R_IMGO				BIT(0)
> +#define R_RRZO				BIT(1)
> +#define R_AAO				BIT(3)
> +#define R_AFO				BIT(4)
> +#define R_LCSO				BIT(5)
> +#define R_PDO				BIT(6)
> +#define R_LMVO				BIT(7)
> +#define R_FLKO				BIT(8)
> +#define R_RSSO				BIT(9)
> +#define R_PSO				BIT(10)
> +
> +#define CQ_BUFFER_COUNT		3
> +#define IRQ_DATA_BUF_SIZE		4
> +#define CQ_ADDRESS_OFFSET		0x640
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> +			"fbc1(0x%x), fbc2(0x%x)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +/* State for struct mtk_isp_p1_ctx: composer_state */
> +enum  {
> +	SCP_ON = 0,
> +	SCP_OFF
> +};

Hmm, looks like bool.

> +
> +enum {
> +	IMG_FMT_UNKNOWN		= 0x0000,
> +	IMG_FMT_RAW_START	= 0x2200,
> +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> +	IMG_FMT_BAYER10,
> +	IMG_FMT_BAYER12,
> +	IMG_FMT_BAYER14,
> +	IMG_FMT_FG_BAYER8,
> +	IMG_FMT_FG_BAYER10,
> +	IMG_FMT_FG_BAYER12,
> +	IMG_FMT_FG_BAYER14,
> +};
> +
> +enum {
> +	RAW_PXL_ID_B = 0,
> +	RAW_PXL_ID_GB,
> +	RAW_PXL_ID_GR,
> +	RAW_PXL_ID_R
> +};

Please use macros with explicitly assigned values for hardware/firmware ABI
definitions.

> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};

Please document the structs above using kerneldoc.

> +
> +/*
> + * struct isp_device - the ISP device information
> + *
> + * @dev: Pointer to struct device
> + * @regs: Camera ISP base register address
> + * @spinlock_irq: Used to protect register read/write data
> + * @current_frame: Current frame sequence number, set when SOF
> + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> + * @sof_count: The accumulated SOF counter
> + * @isp_hw_module: Identity camera A or B
> + *
> + */
> +struct isp_device {

mtk_isp_device?

> +	struct device *dev;
> +	void __iomem *regs;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +/*
> + * struct mtk_isp_p1_ctx - the ISP device information
> + *
> + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> + * @composer_tx_thread: TX Thread for SCP data tranmission
> + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> + * @ipi_occupied: The total number of SCP TX data has beent sent
> + * @scp_state: The state of SCP control
> + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> + * @composed_frame_id: The ack. frame sequence by SCP
> + * @composer_deinit_thread: The de-initialized thread
> + * @p1_enqueue_list: Queue for ISP frame buffers
> + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> + * @irq_data_start: Start index of irq_event_datas ring buffer
> + * @irq_data_end: End index of irq_event_datas ring buffer
> + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> + * @scp_mem_pa: DMA address for SCP device
> + * @scp_mem_iova: DMA address for ISP HW DMA devices
> + * @frame_seq_no: Sequence number for ISP frame buffer
> + * @isp_hw_module: Active camera HW module
> + * @num_clks: The number of driver's clock
> + * @clk_list: The list of clock data
> + * @lock: Lock to protect context operations
> + *
> + */
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	atomic_t ipi_occupied;
> +	atomic_t scp_state;
> +	atomic_t composing_frame;
> +	atomic_t composed_frame_id;
> +	struct isp_thread composer_deinit_thread;
> +	struct isp_queue p1_enqueue_list;
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */

Already documented in kerneldoc.

> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;

Not documented above.

> +	unsigned int num_clks;
> +	struct clk_bulk_data *clk_list;
> +	struct mutex lock; /* Protect context operations */

Already documented in kerneldoc.

> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct mtk_cam_dev cam_dev;
> +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> +};

Please document in a kerneldoc comment.

> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));

No need to cast. And, I don't think we need a function for this, just call
dev_get_drvdata() directly.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-10  9:56       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
>  4 files changed, 1457 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..9e59a6bfc6b7
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,126 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT			BIT(0)
> +#define CMOS_EN_BIT			BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST			BIT(0)
> +#define HW_PASS1_DON_ST		BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define SW_PASS1_DON_ST		BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST			BIT(4)
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +#define PSO_DONE_ST			BIT(14)
> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM		( \
> +					VS_INT_ST |\
> +					SOF_INT_ST |\
> +					HW_PASS1_DON_ST |\
> +					SW_PASS1_DON_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM		( \
> +					SOF_INT_ST |\
> +					SW_PASS1_DON_ST |\
> +					HW_PASS1_DON_ST |\
> +					VS_INT_ST |\
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					RRZO_ERR_ST |\
> +					AFO_ERR_ST |\
> +					IMGO_ERR_ST |\
> +					AAO_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM		( \
> +					AAO_DONE_ST |\
> +					AFO_DONE_ST)

Could we define the values next to the addresses of registers they apply to?
Also without the _BIT suffix and with the values prefixed with register
names. For example:

#define REG_TG_SEN_MODE		        0x0230
#define TG_SEN_MODE_CMOS_EN		BIT(0)

#define REG_TG_VF_CON			0x0234
#define TG_VF_CON_VFDATA_EN		BIT(0)

> +
> +/* Status check */
> +#define REG_CTL_EN			0x0004
> +#define REG_CTL_DMA_EN			0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2			0x0018
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_VF_CON			0x0234
> +
> +#define REG_IMGO_BASE_ADDR		0x1020
> +#define REG_RRZO_BASE_ADDR		0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT		0x1360
> +#define REG_RRZO_ERR_STAT		0x1364
> +#define REG_AAO_ERR_STAT		0x1368
> +#define REG_AFO_ERR_STAT		0x136c
> +#define REG_LCSO_ERR_STAT		0x1370
> +#define REG_UFEO_ERR_STAT		0x1374
> +#define REG_PDO_ERR_STAT		0x1378
> +#define REG_BPCI_ERR_STAT		0x137c
> +#define REG_LSCI_ERR_STAT		0x1384
> +#define REG_PDI_ERR_STAT		0x138c
> +#define REG_LMVO_ERR_STAT		0x1390
> +#define REG_FLKO_ERR_STAT		0x1394
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +#define REG_HW_FRAME_NUM		0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX		0x14dc
> +#define REG_META1_VB2_INDEX		0x151c

I don't believe these registers are really for VB2 indexes.

> +
> +/* FBC */
> +#define REG_AAO_FBC_STATUS		0x013c
> +#define REG_AFO_FBC_STATUS		0x0134
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..c5a3babed69d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +#include <media/v4l2-event.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);

Please move below. Just above the platform_driver struct where it's used.

> +
> +/* List of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +};

Please move inside mtk_isp_probe, as a static const local variable. That
could also let you shorten the name, to clk_names for example.

> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +					 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;

nit: You can just initialize the structure in the declaration.

> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +}
> +
> +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> +				   unsigned int request_fd,
> +				   unsigned int frame_seq_no,
> +				   struct list_head *list_buf,
> +				   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	u64    timestamp;

Too many spaces between u64 and timestamp.

> +
> +	if (!cam_dev->streaming)
> +		return;
> +
> +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> +		__func__, request_fd, frame_seq_no, state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	timestamp = ktime_get_ns();
> +	list_for_each_entry_safe(buf, b0, list_buf, list) {

nit: s/b0/buf_prev/

> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = timestamp;
> +		buf->vbb.sequence = frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

Any buffer that is not active shouldn't be on this list. If it is then it's
a bug somewhere else in the driver. Could be possibly related to the request
handling issues I pointed out in another comment.

> +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,

dequeue

> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct vb2_buffer *vb;
> +
> +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> +		return;
> +
> +	spin_lock(&node->slock);
> +	b = list_first_entry(&node->pending_list,
> +			     struct mtk_cam_dev_buffer,
> +			     list);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		vb = &b->vbb.vb2_buf;
> +		if (!vb->vb2_queue->uses_requests &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vb->timestamp = ktime_get_ns();
> +			b->vbb.sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +			list_del(&b->list);
> +			break;
> +		}
> +	}
> +	spin_unlock(&node->slock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,

dequeue

> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_warn(dev,
> +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> +				 framejob->frame_seq_no,
> +				 atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		} else {
> +			break;
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)

dequeue

> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> +		    event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int addr_offset;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	isp_dev->sof_count += 1;
> +
> +	if (cq_num <= frame_num) {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> +		return cq_num;
> +	}
> +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> +
> +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(isp_dev->dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> +		cq_num, frame_num, addr_offset);
> +
> +	return cq_num;
> +}
> +
> +static void irq_handle_notify_event(struct isp_device *isp_dev,
> +				    unsigned int irq_status,
> +				    unsigned int dma_status,
> +				    bool sof_only)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned long flags;
> +	int i;
> +
> +	if (irq_status & VS_INT_ST) {
> +		/* Notify specific HW events to user space */
> +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> +					     isp_dev->current_frame);

Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
definition of the V4L2_EVENT_FRAME_SYNC event at
https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html

> +		dev_dbg(dev,
> +			"frame sync is sent:%d:%d\n",
> +			isp_dev->sof_count,
> +			isp_dev->current_frame);
> +		if (sof_only)
> +			return;

If this function can be called only to perform this block, perhaps it should
be split into two functions?

Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
irq_status? Is it expected that in such case the other part of the function
is executed?

> +	}
> +
> +	if (irq_status & SW_PASS1_DON_ST) {
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask =
> +		(irq_status & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask =
> +		(dma_status & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);

I can see that all isp_deque_work() does is returning buffers to vb2. I
don't think we need this intricate system to do that, as we could just do
it inside the interrupt handler, in isp_irq_cam() directly.

> +
> +	dev_dbg(dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irq_status & INT_ST_MASK_CAM),
> +		(dma_status & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cam_idx, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned int aao_fbc, afo_fbc;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev.streaming)
> +		return IRQ_HANDLED;

This shouldn't be needed. The driver needs to unmask the interrupts in
hardware registers when it starts streaming and mask them when it stops.
Note that I mean the P1 hardware registers, not disable_irq(), which
shouldn't be used.

> +
> +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);

Hmm, reading vb2 buffer index from hardware registers? Was this hardware
designed exclusively for V4L2? ;)

Jokes aside, how does the hardware know V4L2 buffer indexes?

> +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> +		return IRQ_HANDLED;

Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
get an interrupt without any unmasked hardware IRQs active in the status,
that's an error somewhere and we should return IRQ_NONE.

> +
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +
> +	/* Sof, done order check */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue frame */
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> +		isp_dev->current_frame = hw_frame_num;

What exactly is hw_frame_num? Shouldn't we assign it before notifying the
event?

> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irq_status & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> +	}

The if and else blocks do almost the same things just in different order. Is
it really expected?

> +
> +	if (irq_status & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x\n",
> +			irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irq_status & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,

Please just put that string here, otherwise the reader would have no idea
what message is being printed here.

> +			'A' + cam_idx,
> +			isp_dev->sof_count,
> +			irq_status,
> +			dma_status,
> +			hw_frame_num,
> +			cq_num,
> +			aao_fbc,
> +			afo_fbc);

nit: No need to put each argument in its own line.

> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}

This look-up should be done once in probe(). Only the rproc_boot() should
happen dynamically.

> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +	mutex_init(&isp_ctx->lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +
> +	return 0;

Everything here looks like something that should be done once in probe. I
also don't see a point of having a separate mtk_isp_p1_ctx struct for the
data above. It could be just located in p1_dev, at least for now.

If we end up implementing support for multiple contexts, we could isolate
per-context data then, once we know what's really per-context. For now,
let's keep it simple.

> +}
> +
> +static int isp_uninit_context(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	if (isp_ctx->isp_deque_thread.thread) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	mutex_destroy(&isp_ctx->lock);
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int enabled_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enabled_dma_ports = 0;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		if (cam_dev->vdev_nodes[i].enabled)
> +			enabled_dma_ports |=
> +				cam_dev->vdev_nodes[i].desc.dma_port;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> +
> +	return enabled_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return RAW_PXL_ID_R;
> +	default:

Could we fail here instead?

> +		return RAW_PXL_ID_B;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return IMG_FMT_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return IMG_FMT_FG_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:

Could we fail here instead, so that we don't mask some potential errors?

> +		return 10;
> +	}
> +}
> +
> +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> +			   const struct v4l2_format *in_fmt,
> +			   const struct v4l2_subdev_format *sd_format)
> +{
> +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> +
> +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;

Please group operations on the same field together, i.e. remove the blank
line above size.stride and add one blank line above size.w.

> +
> +	out_fmt->crop.left = 0x0;
> +	out_fmt->crop.top = 0x0;
> +

Remove the blank line.

> +	out_fmt->crop.width = sd_format->format.width;
> +	out_fmt->crop.height = sd_format->format.height;
> +
> +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> +		  "img out:%d:%d in:%d:%d",
> +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> +		  out_fmt->crop.width, out_fmt->crop.height);

We should check this earlier and fail the streaming start if there is a
mismatch between sensor and video node configuration.

> +
> +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> +		out_fmt->pixel_byte,
> +		out_fmt->img_fmt);
> +	dev_dbg(dev,
> +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = isp_setup_scp_rproc(p1_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = isp_init_context(p1_dev);
> +	if (ret)
> +		return ret;

The above function doesn't really seem to be related to power management.
Should it be called from subdev stream on?

> +
> +	ret = isp_composer_init(dev);
> +	if (ret)
> +		goto composer_err;

This also doesn't seem to be related to power management.

> +
> +	pm_runtime_get_sync(dev);
> +
> +	/* ISP HW INIT */
> +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;

There is some amount of code handling various values of isp_hw_module in
this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
code that can't be tested. Please either add support for all the indexes in
the driver or simplify all the code to just handle CAM_B.

> +	/* Use pure RAW as default HW path */
> +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> +
> +	isp_composer_hw_init(dev);
> +	/* Check enabled DMAs which is configured by media setup */
> +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));

Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
necessary?

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +
> +composer_err:
> +	isp_uninit_context(dev);
> +
> +	return ret;
> +}
> +
> +int mtk_isp_power_release(struct device *dev)
> +{
> +	isp_composer_hw_deinit(dev);
> +	isp_uninit_context(dev);

These two don't seem to be related to power either.

Instead, I don't see anything that could undo the rproc_boot() operation
here.

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_fmt;
> +	unsigned int enabled_dma_ports;
> +	struct v4l2_format *img_fmt;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> +
> +	/* Sensor config */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> +

Unnecessary blank line.

> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	dev_dbg(dev,
> +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_fmt.format.width, sd_fmt.format.height,
> +		sd_fmt.format.code, sd_fmt.format.field,
> +		sd_fmt.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* Fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = 0x1;
> +	/* Support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;

Why hexadecimal numerals? Also, what's the meaning of these values? For
anything boolean, you could just use true and false as a substite of 0 and
1. For anything that has more values, please define the values using macros.

> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_fmt.format.code);
> +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;

Move the other crop settings from above to here.

> +
> +	config_param.cfg_main_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {

No need for the == R_IMGO part.

> +		config_param.cfg_main_param.bypass = 0;
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	config_param.cfg_resize_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {

Ditto.

> +		config_param.cfg_resize_param.bypass = 0;
> +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;

Should image DMAs be masked out of this bitmap?

> +
> +	isp_composer_hw_config(dev, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> +		     struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> +}
> +
> +void mtk_isp_req_flush_buffers(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_queue_job *job, *j0;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> +
> +	if (!atomic_read(&p1_list->queue_cnt))
> +		return;

Do we need this explicit check? The code below wouldn't do anything if there
isn't anything in the list. IMHO we could even remove queue_cnt completely.

> +
> +	spin_lock(&p1_list->lock);
> +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {

nit: s/j0/job_prev/

> +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {

nit: s/b0/buf_pref/

Also, we should be able to replace this with iterating over the generic list
of request objects, rather than this internal list.

> +			list_del(&buf->list);
> +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

It shouldn't be possible to happen. If you see this check failing, that
means a problem somewhere else in the driver.

> +				vb2_buffer_done(&buf->vbb.vb2_buf,
> +						VB2_BUF_STATE_ERROR);
> +		}
> +		list_del(&job->list_entry);
> +		atomic_dec(&p1_list->queue_cnt);
> +		kfree(job);
> +	}
> +	spin_unlock(&p1_list->lock);
> +}
> +
> +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct media_request_object *obj, *obj_safe;
> +	struct vb2_buffer *vb;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);

This allocation shouldn't be needed. The structure should be already a part
of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
even say that the contents of this struct should be just moved to that one
to avoid overabstracting.

> +	memset(framejob, 0, sizeof(*framejob));

Putting the above comment aside, kzalloc() already zeroes the memory.

> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;

How is this synchronized with the sof_count increment in irq_handle_sof()?

> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +		vb = container_of(obj, struct vb2_buffer, req_obj);

We should check that the object type before assuming it's a vb2_buffer by
calling vb2_request_object_is_buffer().

> +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +		framejob->request_fd = buf->vbb.request_fd;

We shouldn't use request_fd as the key here. We already have req here, which
is the right key to use.

That said, I can see framejob->request_fd only used for printing a debugging
message in mtk_cam_dev_job_finish(). The request API core should already
print something for us once a request is completed, so perhaps that isn't
needed?

> +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> +		list_add_tail(&buf->list, &framejob->list_buf);

Why do we need this private list? We could just call exactly the same
list_for_each() over the request objects.

> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);

We already have a list head in mtk_cam_dev_request.

> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		framejob->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s\n", __func__);

dev_dbg()

> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> +				      p1_dev->isp_ctx.clk_list);
> +	if (ret)
> +		goto clk_err;
> +
> +	return 0;
> +
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);

clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
this would disable the clocks second time.

> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);

dev_dbg()

> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);
> +}

There is no point in having wrapper functions to just call one function
inside. Please just call clk_bulk_*() directly.

> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check if the device isn't already runtime suspended. If it is, we
don't have to do anything here and can just return.


We also need to ensure that no new requests are queued to the hardware at
this point. This could be done by replacing any of the kthreads with
workqueues and making all of the workqueues freezable.

> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* Disable view finder */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, the TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}

Are you sure this is the right handling? We need to make sure the hardware
finishes processing the current frame and stops.

> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check runtime PM status here as well, because if the device was
suspended, there is nothing to do here.

> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev.streaming) {
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}

Does the hardware keep all the state, e.g. queued buffers, during suspend?
Would the code above continue all the capture from the next buffer, as
queued by the userspace before the suspend?

> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq;
> +	int ret;
> +	unsigned int i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;

Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
reference to platform_device is very useful later in the code.

> +
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {

I think we want to start from 0 here?

> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;

p1_dev already includes a pointer to this.

> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> +			i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)

devm_ioremap_resource() returns ERR_PTR() not NULL on error.

> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* Support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* Reg & interrupts index is shifted with 1  */

The reader can already see that it's shifted from the code below. The
comment should say _why_ it is shifted.

> +			irq = platform_get_irq(pdev, i - 1);

The bindings have 3 IRQs, but we only seem to request 2 here. Is that
expected?

> +			if (irq) {

Please invert this if, so that it bails out on error. Also, this should
check for negative errors codes, not 0.

> +				ret = devm_request_irq(dev, irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),

Use dev_name().

> +						       (void *)isp_dev);

No need to cast to void *.

> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						irq);
> +					return ret;
> +				}
> +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> +					irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}

We might want to move out the body of this loop to a separate function to
keep this function shorter.

> +
> +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_ctx.clk_list =

nit: "list" is the term commonly used for the list data structure. There is
also a convention to call the length of array XXX num_XXX, so how about
clks and num_clks?

> +		devm_kcalloc(dev,
> +			     p1_dev->isp_ctx.num_clks,
> +			     sizeof(*p1_dev->isp_ctx.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_ctx.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];

Shouldn't this be clk_list[i].id?

> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_ctx.num_clks,
> +				p1_dev->isp_ctx.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> +		goto err_init;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret)
> +		goto err_init;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +
> +err_init:
> +	if (p1_dev->cam_dev.smem_dev)
> +		device_unregister(p1_dev->cam_dev.smem_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)

For V4L2 drivers system and runtime PM ops would normally be completely
different. Runtime PM ops would be called when the hardware is idle already
or is about to become active. System PM ops would be called at system power
state change and the hardware might be both idle or active. Please also see
my comments to mtk_isp_suspend() and mtk_isp_resume() above.

> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",

Shouldn't this be something like mtk-cam-p1? Please make sure this
reasonably consistent with other drivers.

> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");

Mediatek Camera P1 ISP driver? Please make sure this is reasonably
consistent with other drivers (SenInf, DIP, FD).

> +MODULE_LICENSE("GPL");

GPL v2

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-scp.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define CAM_A_MAX_WIDTH		3328
> +#define CAM_A_MAX_HEIGHT		2496
> +#define CAM_B_MAX_WIDTH		5376
> +#define CAM_B_MAX_HEIGHT		4032
> +
> +#define CAM_MIN_WIDTH			80
> +#define CAM_MIN_HEIGHT			60
> +
> +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define R_IMGO				BIT(0)
> +#define R_RRZO				BIT(1)
> +#define R_AAO				BIT(3)
> +#define R_AFO				BIT(4)
> +#define R_LCSO				BIT(5)
> +#define R_PDO				BIT(6)
> +#define R_LMVO				BIT(7)
> +#define R_FLKO				BIT(8)
> +#define R_RSSO				BIT(9)
> +#define R_PSO				BIT(10)
> +
> +#define CQ_BUFFER_COUNT		3
> +#define IRQ_DATA_BUF_SIZE		4
> +#define CQ_ADDRESS_OFFSET		0x640
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> +			"fbc1(0x%x), fbc2(0x%x)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +/* State for struct mtk_isp_p1_ctx: composer_state */
> +enum  {
> +	SCP_ON = 0,
> +	SCP_OFF
> +};

Hmm, looks like bool.

> +
> +enum {
> +	IMG_FMT_UNKNOWN		= 0x0000,
> +	IMG_FMT_RAW_START	= 0x2200,
> +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> +	IMG_FMT_BAYER10,
> +	IMG_FMT_BAYER12,
> +	IMG_FMT_BAYER14,
> +	IMG_FMT_FG_BAYER8,
> +	IMG_FMT_FG_BAYER10,
> +	IMG_FMT_FG_BAYER12,
> +	IMG_FMT_FG_BAYER14,
> +};
> +
> +enum {
> +	RAW_PXL_ID_B = 0,
> +	RAW_PXL_ID_GB,
> +	RAW_PXL_ID_GR,
> +	RAW_PXL_ID_R
> +};

Please use macros with explicitly assigned values for hardware/firmware ABI
definitions.

> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};

Please document the structs above using kerneldoc.

> +
> +/*
> + * struct isp_device - the ISP device information
> + *
> + * @dev: Pointer to struct device
> + * @regs: Camera ISP base register address
> + * @spinlock_irq: Used to protect register read/write data
> + * @current_frame: Current frame sequence number, set when SOF
> + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> + * @sof_count: The accumulated SOF counter
> + * @isp_hw_module: Identity camera A or B
> + *
> + */
> +struct isp_device {

mtk_isp_device?

> +	struct device *dev;
> +	void __iomem *regs;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +/*
> + * struct mtk_isp_p1_ctx - the ISP device information
> + *
> + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> + * @composer_tx_thread: TX Thread for SCP data tranmission
> + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> + * @ipi_occupied: The total number of SCP TX data has beent sent
> + * @scp_state: The state of SCP control
> + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> + * @composed_frame_id: The ack. frame sequence by SCP
> + * @composer_deinit_thread: The de-initialized thread
> + * @p1_enqueue_list: Queue for ISP frame buffers
> + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> + * @irq_data_start: Start index of irq_event_datas ring buffer
> + * @irq_data_end: End index of irq_event_datas ring buffer
> + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> + * @scp_mem_pa: DMA address for SCP device
> + * @scp_mem_iova: DMA address for ISP HW DMA devices
> + * @frame_seq_no: Sequence number for ISP frame buffer
> + * @isp_hw_module: Active camera HW module
> + * @num_clks: The number of driver's clock
> + * @clk_list: The list of clock data
> + * @lock: Lock to protect context operations
> + *
> + */
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	atomic_t ipi_occupied;
> +	atomic_t scp_state;
> +	atomic_t composing_frame;
> +	atomic_t composed_frame_id;
> +	struct isp_thread composer_deinit_thread;
> +	struct isp_queue p1_enqueue_list;
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */

Already documented in kerneldoc.

> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;

Not documented above.

> +	unsigned int num_clks;
> +	struct clk_bulk_data *clk_list;
> +	struct mutex lock; /* Protect context operations */

Already documented in kerneldoc.

> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct mtk_cam_dev cam_dev;
> +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> +};

Please document in a kerneldoc comment.

> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));

No need to cast. And, I don't think we need a function for this, just call
dev_get_drvdata() directly.

Best regards,
Tomasz


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-10  9:56       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other functions.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
>  4 files changed, 1457 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..9e59a6bfc6b7
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,126 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _CAM_REGS_H
> +#define _CAM_REGS_H
> +
> +/* TG Bit Mask */
> +#define VFDATA_EN_BIT			BIT(0)
> +#define CMOS_EN_BIT			BIT(0)
> +
> +/* normal signal bit */
> +#define VS_INT_ST			BIT(0)
> +#define HW_PASS1_DON_ST		BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define SW_PASS1_DON_ST		BIT(30)
> +
> +/* err status bit */
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +
> +/* CAM DMA done status */
> +#define FLKO_DONE_ST			BIT(4)
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +#define PSO_DONE_ST			BIT(14)
> +
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM		( \
> +					VS_INT_ST |\
> +					SOF_INT_ST |\
> +					HW_PASS1_DON_ST |\
> +					SW_PASS1_DON_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* IRQ Signal Log Mask */
> +#define INT_ST_LOG_MASK_CAM		( \
> +					SOF_INT_ST |\
> +					SW_PASS1_DON_ST |\
> +					HW_PASS1_DON_ST |\
> +					VS_INT_ST |\
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					RRZO_ERR_ST |\
> +					AFO_ERR_ST |\
> +					IMGO_ERR_ST |\
> +					AAO_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +/* DMA Event Notification Mask */
> +#define DMA_ST_MASK_CAM		( \
> +					AAO_DONE_ST |\
> +					AFO_DONE_ST)

Could we define the values next to the addresses of registers they apply to?
Also without the _BIT suffix and with the values prefixed with register
names. For example:

#define REG_TG_SEN_MODE		        0x0230
#define TG_SEN_MODE_CMOS_EN		BIT(0)

#define REG_TG_VF_CON			0x0234
#define TG_VF_CON_VFDATA_EN		BIT(0)

> +
> +/* Status check */
> +#define REG_CTL_EN			0x0004
> +#define REG_CTL_DMA_EN			0x0008
> +#define REG_CTL_FMT_SEL		0x0010
> +#define REG_CTL_EN2			0x0018
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +
> +#define REG_TG_SEN_MODE		0x0230
> +#define REG_TG_VF_CON			0x0234
> +
> +#define REG_IMGO_BASE_ADDR		0x1020
> +#define REG_RRZO_BASE_ADDR		0x1050
> +
> +/* Error status log */
> +#define REG_IMGO_ERR_STAT		0x1360
> +#define REG_RRZO_ERR_STAT		0x1364
> +#define REG_AAO_ERR_STAT		0x1368
> +#define REG_AFO_ERR_STAT		0x136c
> +#define REG_LCSO_ERR_STAT		0x1370
> +#define REG_UFEO_ERR_STAT		0x1374
> +#define REG_PDO_ERR_STAT		0x1378
> +#define REG_BPCI_ERR_STAT		0x137c
> +#define REG_LSCI_ERR_STAT		0x1384
> +#define REG_PDI_ERR_STAT		0x138c
> +#define REG_LMVO_ERR_STAT		0x1390
> +#define REG_FLKO_ERR_STAT		0x1394
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* ISP command */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +#define REG_HW_FRAME_NUM		0x13b8
> +
> +/* META */
> +#define REG_META0_VB2_INDEX		0x14dc
> +#define REG_META1_VB2_INDEX		0x151c

I don't believe these registers are really for VB2 indexes.

> +
> +/* FBC */
> +#define REG_AAO_FBC_STATUS		0x013c
> +#define REG_AFO_FBC_STATUS		0x0134
> +
> +#endif	/* _CAM_REGS_H */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..c5a3babed69d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,1087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/cdev.h>
> +#include <linux/compat.h>
> +#include <linux/fs.h>
> +#include <linux/interrupt.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/ktime.h>
> +#include <linux/module.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/platform_device.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched/clock.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +#include <media/v4l2-event.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-regs.h"
> +#include "mtk_cam-smem.h"
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);

Please move below. Just above the platform_driver struct where it's used.

> +
> +/* List of clocks required by isp cam */
> +static const char * const mtk_isp_clks[] = {
> +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +};

Please move inside mtk_isp_probe, as a static const local variable. That
could also let you shorten the name, to clk_names for example.

> +
> +static void isp_dump_dma_status(struct isp_device *isp_dev)
> +{
> +	dev_err(isp_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(isp_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> +}
> +
> +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +					 __u32 frame_seq_no)
> +{
> +	struct v4l2_event event;
> +
> +	memset(&event, 0, sizeof(event));
> +	event.type = V4L2_EVENT_FRAME_SYNC;
> +	event.u.frame_sync.frame_sequence = frame_seq_no;

nit: You can just initialize the structure in the declaration.

> +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> +}
> +
> +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> +				   unsigned int request_fd,
> +				   unsigned int frame_seq_no,
> +				   struct list_head *list_buf,
> +				   enum vb2_buffer_state state)
> +{
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	u64    timestamp;

Too many spaces between u64 and timestamp.

> +
> +	if (!cam_dev->streaming)
> +		return;
> +
> +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> +		__func__, request_fd, frame_seq_no, state);
> +
> +	/*
> +	 * Set the buffer's VB2 status so that the user can dequeue
> +	 * the buffer.
> +	 */
> +	timestamp = ktime_get_ns();
> +	list_for_each_entry_safe(buf, b0, list_buf, list) {

nit: s/b0/buf_prev/

> +		list_del(&buf->list);
> +		buf->vbb.vb2_buf.timestamp = timestamp;
> +		buf->vbb.sequence = frame_seq_no;
> +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

Any buffer that is not active shouldn't be on this list. If it is then it's
a bug somewhere else in the driver. Could be possibly related to the request
handling issues I pointed out in another comment.

> +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +static void isp_deque_frame(struct isp_p1_device *p1_dev,

dequeue

> +			    unsigned int node_id, int vb2_index,
> +			    int frame_seq_no)
> +{
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> +	struct mtk_cam_dev_buffer *b, *b0;
> +	struct vb2_buffer *vb;
> +
> +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> +		return;
> +
> +	spin_lock(&node->slock);
> +	b = list_first_entry(&node->pending_list,
> +			     struct mtk_cam_dev_buffer,
> +			     list);
> +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> +		vb = &b->vbb.vb2_buf;
> +		if (!vb->vb2_queue->uses_requests &&
> +		    vb->index == vb2_index &&
> +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> +			vb->timestamp = ktime_get_ns();
> +			b->vbb.sequence = frame_seq_no;
> +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +			list_del(&b->list);
> +			break;
> +		}
> +	}
> +	spin_unlock(&node->slock);
> +}
> +
> +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,

dequeue

> +				    int frame_seq_no)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_job *framejob, *tmp;
> +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> +
> +	/* Match dequeue work and enqueue frame */
> +	spin_lock(&p1_enqueue_list->lock);
> +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> +				 list_entry) {
> +		dev_dbg(dev,
> +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> +			__func__,
> +			framejob->frame_seq_no, frame_seq_no);
> +		/* Match by the en-queued request number */
> +		if (framejob->frame_seq_no == frame_seq_no) {
> +			/* Pass to user space */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_DONE);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_dbg(dev,
> +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> +				framejob->frame_seq_no,
> +				atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only when frame ready */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +			break;
> +		} else if (framejob->frame_seq_no < frame_seq_no) {
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_finish(isp_ctx,
> +					       framejob->request_fd,
> +					       framejob->frame_seq_no,
> +					       &framejob->list_buf,
> +					       VB2_BUF_STATE_ERROR);
> +			atomic_dec(&p1_enqueue_list->queue_cnt);
> +			dev_warn(dev,
> +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> +				 framejob->frame_seq_no,
> +				 atomic_read(&p1_enqueue_list->queue_cnt));
> +
> +			/* Remove only drop frame */
> +			list_del(&framejob->list_entry);
> +			kfree(framejob);
> +		} else {
> +			break;
> +		}
> +	}
> +	spin_unlock(&p1_enqueue_list->lock);
> +}
> +
> +static int isp_deque_work(void *data)

dequeue

> +{
> +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_cam_dev_stat_event_data event_data;
> +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> +	unsigned long flags;
> +	int ret, i;
> +
> +	while (1) {
> +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> +					       (atomic_read(irq_data_end) !=
> +					       atomic_read(irq_data_start)) ||
> +					       kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +		i = atomic_read(&isp_ctx->irq_data_start);
> +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> +		       sizeof(event_data));
> +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> +		    event_data.dma_status_mask & AAO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +		}
> +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_0,
> +					event_data.meta0_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_frame(p1_dev,
> +					MTK_CAM_P1_META_OUT_1,
> +					event_data.meta1_vb2_index,
> +					event_data.frame_seq_no);
> +			isp_deque_request_frame(p1_dev,
> +						event_data.frame_seq_no);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int irq_handle_sof(struct isp_device *isp_dev,
> +			  dma_addr_t base_addr,
> +			  unsigned int frame_num)
> +{
> +	unsigned int addr_offset;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> +
> +	isp_dev->sof_count += 1;
> +
> +	if (cq_num <= frame_num) {
> +		dev_dbg(isp_dev->dev,
> +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> +			cq_num, frame_num);
> +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> +		return cq_num;
> +	}
> +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> +
> +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(isp_dev->dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> +		cq_num, frame_num, addr_offset);
> +
> +	return cq_num;
> +}
> +
> +static void irq_handle_notify_event(struct isp_device *isp_dev,
> +				    unsigned int irq_status,
> +				    unsigned int dma_status,
> +				    bool sof_only)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned long flags;
> +	int i;
> +
> +	if (irq_status & VS_INT_ST) {
> +		/* Notify specific HW events to user space */
> +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> +					     isp_dev->current_frame);

Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
definition of the V4L2_EVENT_FRAME_SYNC event at
https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html

> +		dev_dbg(dev,
> +			"frame sync is sent:%d:%d\n",
> +			isp_dev->sof_count,
> +			isp_dev->current_frame);
> +		if (sof_only)
> +			return;

If this function can be called only to perform this block, perhaps it should
be split into two functions?

Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
irq_status? Is it expected that in such case the other part of the function
is executed?

> +	}
> +
> +	if (irq_status & SW_PASS1_DON_ST) {
> +		/* Notify TX thread to send if TX frame is blocked */
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +	}
> +
> +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> +	i = atomic_read(&isp_ctx->irq_data_end);
> +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> +	isp_ctx->irq_event_datas[i].irq_status_mask =
> +		(irq_status & INT_ST_MASK_CAM);
> +	isp_ctx->irq_event_datas[i].dma_status_mask =
> +		(dma_status & DMA_ST_MASK_CAM);
> +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> +
> +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);

I can see that all isp_deque_work() does is returning buffers to vb2. I
don't think we need this intricate system to do that, as we could just do
it inside the interrupt handler, in isp_irq_cam() directly.

> +
> +	dev_dbg(dev,
> +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> +		__func__,
> +		(irq_status & INT_ST_MASK_CAM),
> +		(dma_status & DMA_ST_MASK_CAM),
> +		isp_dev->current_frame,
> +		isp_dev->meta0_vb2_index,
> +		isp_dev->meta1_vb2_index);
> +}
> +
> +irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct isp_device *isp_dev = (struct isp_device *)data;
> +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = isp_dev->dev;
> +	unsigned int cam_idx, cq_num, hw_frame_num;
> +	unsigned int meta0_vb2_index, meta1_vb2_index;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned int aao_fbc, afo_fbc;
> +	unsigned long flags;
> +
> +	/* Check the streaming is off or not */
> +	if (!p1_dev->cam_dev.streaming)
> +		return IRQ_HANDLED;

This shouldn't be needed. The driver needs to unmask the interrupts in
hardware registers when it starts streaming and mask them when it stops.
Note that I mean the P1 hardware registers, not disable_irq(), which
shouldn't be used.

> +
> +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> +	cq_num = 0;
> +
> +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);

Hmm, reading vb2 buffer index from hardware registers? Was this hardware
designed exclusively for V4L2? ;)

Jokes aside, how does the hardware know V4L2 buffer indexes?

> +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> +
> +	/* Ignore unnecessary IRQ */
> +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> +		return IRQ_HANDLED;

Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
get an interrupt without any unmasked hardware IRQs active in the status,
that's an error somewhere and we should return IRQ_NONE.

> +
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +
> +	/* Sof, done order check */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> +
> +		/* Notify IRQ event and enqueue frame */
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> +		isp_dev->current_frame = hw_frame_num;

What exactly is hw_frame_num? Shouldn't we assign it before notifying the
event?

> +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> +	} else {
> +		if (irq_status & SOF_INT_ST) {
> +			isp_dev->current_frame = hw_frame_num;
> +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> +		}
> +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> +	}

The if and else blocks do almost the same things just in different order. Is
it really expected?

> +
> +	if (irq_status & SOF_INT_ST)
> +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> +					hw_frame_num);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev,
> +			"raw_int_err:0x%x/0x%x\n",
> +			irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_dump_dma_status(isp_dev);
> +	}
> +
> +	if (irq_status & INT_ST_LOG_MASK_CAM)
> +		dev_dbg(dev, IRQ_STAT_STR,

Please just put that string here, otherwise the reader would have no idea
what message is being printed here.

> +			'A' + cam_idx,
> +			isp_dev->sof_count,
> +			irq_status,
> +			dma_status,
> +			hw_frame_num,
> +			cq_num,
> +			aao_fbc,
> +			afo_fbc);

nit: No need to put each argument in its own line.

> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> +{
> +	phandle rproc_phandle;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> +	if (!p1_dev->scp_pdev) {
> +		dev_err(dev, "Failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> +				   &rproc_phandle);
> +	if (ret) {
> +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> +	if (!p1_dev->rproc_handle) {
> +		dev_err(dev, "fail to get rproc_handle\n");
> +		return -EINVAL;
> +	}

This look-up should be done once in probe(). Only the rproc_boot() should
happen dynamically.

> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		/*
> +		 * Return 0 if downloading firmware successfully,
> +		 * otherwise it is failed
> +		 */
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int isp_init_context(struct isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct device *dev = &p1_dev->pdev->dev;
> +	unsigned int i;
> +
> +	dev_dbg(dev, "init irq work thread\n");
> +	if (!isp_ctx->isp_deque_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread =
> +			kthread_run(isp_deque_work, (void *)p1_dev,
> +				    "isp_deque_work");
> +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> +			dev_err(dev, "unable to alloc kthread\n");
> +			isp_ctx->isp_deque_thread.thread = NULL;
> +			return -ENOMEM;
> +		}
> +	}
> +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> +	mutex_init(&isp_ctx->lock);
> +
> +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> +
> +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> +
> +	atomic_set(&isp_ctx->irq_data_end, 0);
> +	atomic_set(&isp_ctx->irq_data_start, 0);
> +
> +	return 0;

Everything here looks like something that should be done once in probe. I
also don't see a point of having a separate mtk_isp_p1_ctx struct for the
data above. It could be just located in p1_dev, at least for now.

If we end up implementing support for multiple contexts, we could isolate
per-context data then, once we know what's really per-context. For now,
let's keep it simple.

> +}
> +
> +static int isp_uninit_context(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> +
> +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> +	list_for_each_entry_safe(framejob, tmp_framejob,
> +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> +		list_del(&framejob->list_entry);
> +		kfree(framejob);
> +	}
> +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> +
> +	if (isp_ctx->isp_deque_thread.thread) {
> +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> +		isp_ctx->isp_deque_thread.thread = NULL;
> +	}
> +
> +	mutex_destroy(&isp_ctx->lock);
> +
> +	return 0;
> +}
> +
> +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> +{
> +	unsigned int enabled_dma_ports, i;
> +
> +	/* Get the enabled meta DMA ports */
> +	enabled_dma_ports = 0;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		if (cam_dev->vdev_nodes[i].enabled)
> +			enabled_dma_ports |=
> +				cam_dev->vdev_nodes[i].desc.dma_port;
> +
> +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> +
> +	return enabled_dma_ports;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return RAW_PXL_ID_R;
> +	default:

Could we fail here instead?

> +		return RAW_PXL_ID_B;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return IMG_FMT_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +		return IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +		return IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +		return IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +		return IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return IMG_FMT_FG_BAYER14;
> +	default:
> +		return IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_pixel_byte(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_B8:
> +	case V4L2_PIX_FMT_MTISP_F8:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_B10:
> +	case V4L2_PIX_FMT_MTISP_F10:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_B12:
> +	case V4L2_PIX_FMT_MTISP_F12:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_B14:
> +	case V4L2_PIX_FMT_MTISP_F14:
> +		return 14;
> +	default:

Could we fail here instead, so that we don't mask some potential errors?

> +		return 10;
> +	}
> +}
> +
> +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> +			   const struct v4l2_format *in_fmt,
> +			   const struct v4l2_subdev_format *sd_format)
> +{
> +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> +
> +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;

Please group operations on the same field together, i.e. remove the blank
line above size.stride and add one blank line above size.w.

> +
> +	out_fmt->crop.left = 0x0;
> +	out_fmt->crop.top = 0x0;
> +

Remove the blank line.

> +	out_fmt->crop.width = sd_format->format.width;
> +	out_fmt->crop.height = sd_format->format.height;
> +
> +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> +		  "img out:%d:%d in:%d:%d",
> +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> +		  out_fmt->crop.width, out_fmt->crop.height);

We should check this earlier and fail the streaming start if there is a
mismatch between sensor and video node configuration.

> +
> +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> +		out_fmt->pixel_byte,
> +		out_fmt->img_fmt);
> +	dev_dbg(dev,
> +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +}
> +
> +/* ISP P1 interface functions */
> +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> +{
> +	struct device *dev = &cam_dev->pdev->dev;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = isp_setup_scp_rproc(p1_dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = isp_init_context(p1_dev);
> +	if (ret)
> +		return ret;

The above function doesn't really seem to be related to power management.
Should it be called from subdev stream on?

> +
> +	ret = isp_composer_init(dev);
> +	if (ret)
> +		goto composer_err;

This also doesn't seem to be related to power management.

> +
> +	pm_runtime_get_sync(dev);
> +
> +	/* ISP HW INIT */
> +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;

There is some amount of code handling various values of isp_hw_module in
this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
code that can't be tested. Please either add support for all the indexes in
the driver or simplify all the code to just handle CAM_B.

> +	/* Use pure RAW as default HW path */
> +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> +
> +	isp_composer_hw_init(dev);
> +	/* Check enabled DMAs which is configured by media setup */
> +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));

Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
necessary?

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +
> +composer_err:
> +	isp_uninit_context(dev);
> +
> +	return ret;
> +}
> +
> +int mtk_isp_power_release(struct device *dev)
> +{
> +	isp_composer_hw_deinit(dev);
> +	isp_uninit_context(dev);

These two don't seem to be related to power either.

Instead, I don't see anything that could undo the rproc_boot() operation
here.

> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_config(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_config_param config_param;
> +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> +	struct v4l2_subdev_format sd_fmt;
> +	unsigned int enabled_dma_ports;
> +	struct v4l2_format *img_fmt;
> +	int ret;
> +
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> +
> +	isp_ctx->frame_seq_no = 1;
> +	atomic_set(&isp_ctx->composed_frame_id, 0);
> +
> +	/* Get the enabled DMA ports */
> +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> +
> +	/* Sensor config */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> +

Unnecessary blank line.

> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> +		return -EPERM;

return ret?

> +	}
> +
> +	dev_dbg(dev,
> +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> +		ret, sd_fmt.format.width, sd_fmt.format.height,
> +		sd_fmt.format.code, sd_fmt.format.field,
> +		sd_fmt.format.colorspace);
> +
> +	config_param.cfg_in_param.continuous = 0x1;
> +	config_param.cfg_in_param.subsample = 0x0;
> +	/* Fix to one pixel mode in default */
> +	config_param.cfg_in_param.pixel_mode = 0x1;
> +	/* Support normal pattern in default */
> +	config_param.cfg_in_param.data_pattern = 0x0;
> +
> +	config_param.cfg_in_param.crop.left = 0x0;
> +	config_param.cfg_in_param.crop.top = 0x0;

Why hexadecimal numerals? Also, what's the meaning of these values? For
anything boolean, you could just use true and false as a substite of 0 and
1. For anything that has more values, please define the values using macros.

> +
> +	config_param.cfg_in_param.raw_pixel_id =
> +		get_sensor_pixel_id(sd_fmt.format.code);
> +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;

Move the other crop settings from above to here.

> +
> +	config_param.cfg_main_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {

No need for the == R_IMGO part.

> +		config_param.cfg_main_param.bypass = 0;
> +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> +		config_param.cfg_main_param.pure_raw_pack = 1;
> +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	config_param.cfg_resize_param.bypass = 1;
> +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {

Ditto.

> +		config_param.cfg_resize_param.bypass = 0;
> +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> +			       img_fmt, &sd_fmt);
> +	}
> +
> +	/* Configure meta DMAs info. */
> +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;

Should image DMAs be masked out of this bitmap?

> +
> +	isp_composer_hw_config(dev, &config_param);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> +		     struct mtk_cam_dev_buffer *buffer)
> +{
> +	struct mtk_isp_scp_p1_cmd frameparams;
> +
> +	memset(&frameparams, 0, sizeof(frameparams));
> +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> +	frameparams.meta_frame.enabled_dma = dma_port;
> +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> +}
> +
> +void mtk_isp_req_flush_buffers(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_queue_job *job, *j0;
> +	struct mtk_cam_dev_buffer *buf, *b0;
> +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> +
> +	if (!atomic_read(&p1_list->queue_cnt))
> +		return;

Do we need this explicit check? The code below wouldn't do anything if there
isn't anything in the list. IMHO we could even remove queue_cnt completely.

> +
> +	spin_lock(&p1_list->lock);
> +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {

nit: s/j0/job_prev/

> +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {

nit: s/b0/buf_pref/

Also, we should be able to replace this with iterating over the generic list
of request objects, rather than this internal list.

> +			list_del(&buf->list);
> +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)

It shouldn't be possible to happen. If you see this check failing, that
means a problem somewhere else in the driver.

> +				vb2_buffer_done(&buf->vbb.vb2_buf,
> +						VB2_BUF_STATE_ERROR);
> +		}
> +		list_del(&job->list_entry);
> +		atomic_dec(&p1_list->queue_cnt);
> +		kfree(job);
> +	}
> +	spin_unlock(&p1_list->lock);
> +}
> +
> +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	struct p1_frame_param frameparams;
> +	struct mtk_isp_queue_job *framejob;
> +	struct media_request_object *obj, *obj_safe;
> +	struct vb2_buffer *vb;
> +	struct mtk_cam_dev_buffer *buf;
> +
> +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);

This allocation shouldn't be needed. The structure should be already a part
of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
even say that the contents of this struct should be just moved to that one
to avoid overabstracting.

> +	memset(framejob, 0, sizeof(*framejob));

Putting the above comment aside, kzalloc() already zeroes the memory.

> +	memset(&frameparams, 0, sizeof(frameparams));
> +	INIT_LIST_HEAD(&framejob->list_buf);
> +
> +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> +	frameparams.sof_idx =
> +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;

How is this synchronized with the sof_count increment in irq_handle_sof()?

> +	framejob->frame_seq_no = frameparams.frame_seq_no;
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +		vb = container_of(obj, struct vb2_buffer, req_obj);

We should check that the object type before assuming it's a vb2_buffer by
calling vb2_request_object_is_buffer().

> +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +		framejob->request_fd = buf->vbb.request_fd;

We shouldn't use request_fd as the key here. We already have req here, which
is the right key to use.

That said, I can see framejob->request_fd only used for printing a debugging
message in mtk_cam_dev_job_finish(). The request API core should already
print something for us once a request is completed, so perhaps that isn't
needed?

> +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> +		list_add_tail(&buf->list, &framejob->list_buf);

Why do we need this private list? We could just call exactly the same
list_for_each() over the request objects.

> +	}
> +
> +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);

We already have a list head in mtk_cam_dev_request.

> +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> +
> +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> +		framejob->request_fd,
> +		frameparams.frame_seq_no,
> +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> +}
> +
> +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	struct device *dev = &p1_dev->pdev->dev;
> +	int ret;
> +
> +	dev_info(dev, "- %s\n", __func__);

dev_dbg()

> +
> +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> +				      p1_dev->isp_ctx.clk_list);
> +	if (ret)
> +		goto clk_err;
> +
> +	return 0;
> +
> +clk_err:
> +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);

clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
this would disable the clocks second time.

> +	return ret;
> +}
> +
> +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> +{
> +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);

dev_dbg()

> +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> +				   p1_dev->isp_ctx.clk_list);
> +}

There is no point in having wrapper functions to just call one function
inside. Please just call clk_bulk_*() directly.

> +
> +static int mtk_isp_suspend(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check if the device isn't already runtime suspended. If it is, we
don't have to do anything here and can just return.


We also need to ensure that no new requests are queued to the hardware at
this point. This could be done by replacing any of the kthreads with
workqueues and making all of the workqueues freezable.

> +	isp_dev = &p1_dev->isp_devs[module];
> +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +	if (reg_val & VFDATA_EN_BIT) {
> +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> +		/* Disable view finder */
> +		writel((reg_val & (~VFDATA_EN_BIT)),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +		/*
> +		 * After VF enable, the TG frame count will be reset to 0;
> +		 */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val & (~CMOS_EN_BIT)),
> +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> +	}

Are you sure this is the right handling? We need to make sure the hardware
finishes processing the current frame and stops.

> +
> +	disable_sys_clock(p1_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_resume(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	int module = p1_dev->isp_ctx.isp_hw_module;
> +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> +	unsigned int reg_val;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +

We need to check runtime PM status here as well, because if the device was
suspended, there is nothing to do here.

> +	enable_sys_clock(p1_dev);
> +
> +	/* V4L2 stream-on phase & restore HW stream-on status */
> +	if (p1_dev->cam_dev.streaming) {
> +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> +		/* Enable CMOS */
> +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> +		writel((reg_val | CMOS_EN_BIT),
> +		       isp_dev->regs + REG_TG_SEN_MODE);
> +		/* Enable VF */
> +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> +		writel((reg_val | VFDATA_EN_BIT),
> +		       isp_dev->regs + REG_TG_VF_CON);
> +	}

Does the hardware keep all the state, e.g. queued buffers, during suspend?
Would the code above continue all the capture from the next buffer, as
queued by the userspace before the suspend?

> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	struct isp_p1_device *p1_dev;
> +	struct mtk_isp_p1_ctx *isp_ctx;
> +	struct isp_device *isp_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq;
> +	int ret;
> +	unsigned int i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(dev, p1_dev);
> +	isp_ctx = &p1_dev->isp_ctx;
> +	p1_dev->pdev = pdev;

Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
reference to platform_device is very useful later in the code.

> +
> +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {

I think we want to start from 0 here?

> +		isp_dev = &p1_dev->isp_devs[i];
> +		isp_dev->isp_hw_module = i;
> +		isp_dev->dev = dev;

p1_dev already includes a pointer to this.

> +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> +		isp_dev->regs = devm_ioremap_resource(dev, res);
> +
> +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> +			i, (unsigned long)isp_dev->regs);
> +
> +		if (!isp_dev->regs)

devm_ioremap_resource() returns ERR_PTR() not NULL on error.

> +			return PTR_ERR(isp_dev->regs);
> +
> +		/* Support IRQ from ISP_CAM_A_IDX */
> +		if (i >= ISP_CAM_A_IDX) {
> +			/* Reg & interrupts index is shifted with 1  */

The reader can already see that it's shifted from the code below. The
comment should say _why_ it is shifted.

> +			irq = platform_get_irq(pdev, i - 1);

The bindings have 3 IRQs, but we only seem to request 2 here. Is that
expected?

> +			if (irq) {

Please invert this if, so that it bails out on error. Also, this should
check for negative errors codes, not 0.

> +				ret = devm_request_irq(dev, irq,
> +						       isp_irq_cam,
> +						       IRQF_SHARED,
> +						       dev_driver_string(dev),

Use dev_name().

> +						       (void *)isp_dev);

No need to cast to void *.

> +				if (ret) {
> +					dev_err(dev,
> +						"req_irq fail, dev:%s irq=%d\n",
> +						dev->of_node->name,
> +						irq);
> +					return ret;
> +				}
> +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> +					irq, dev_driver_string(dev));
> +			}
> +		}
> +		spin_lock_init(&isp_dev->spinlock_irq);
> +	}

We might want to move out the body of this loop to a separate function to
keep this function shorter.

> +
> +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> +	p1_dev->isp_ctx.clk_list =

nit: "list" is the term commonly used for the list data structure. There is
also a convention to call the length of array XXX num_XXX, so how about
clks and num_clks?

> +		devm_kcalloc(dev,
> +			     p1_dev->isp_ctx.num_clks,
> +			     sizeof(*p1_dev->isp_ctx.clk_list),
> +			     GFP_KERNEL);
> +	if (!p1_dev->isp_ctx.clk_list)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];

Shouldn't this be clk_list[i].id?

> +
> +	ret = devm_clk_bulk_get(dev,
> +				p1_dev->isp_ctx.num_clks,
> +				p1_dev->isp_ctx.clk_list);
> +	if (ret) {
> +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize reserved DMA memory */
> +	ret = mtk_cam_reserved_memory_init(p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> +		goto err_init;
> +	}
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret)
> +		goto err_init;
> +
> +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> +	pm_runtime_enable(dev);
> +
> +	return 0;
> +
> +err_init:
> +	if (p1_dev->cam_dev.smem_dev)
> +		device_unregister(p1_dev->cam_dev.smem_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	pm_runtime_disable(dev);
> +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)

For V4L2 drivers system and runtime PM ops would normally be completely
different. Runtime PM ops would be called when the hardware is idle already
or is about to become active. System PM ops would be called at system power
state change and the hardware might be both idle or active. Please also see
my comments to mtk_isp_suspend() and mtk_isp_resume() above.

> +};
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam",

Shouldn't this be something like mtk-cam-p1? Please make sure this
reasonably consistent with other drivers.

> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Camera ISP driver");

Mediatek Camera P1 ISP driver? Please make sure this is reasonably
consistent with other drivers (SenInf, DIP, FD).

> +MODULE_LICENSE("GPL");

GPL v2

diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..6af3f569664c
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,243 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef __CAMERA_ISP_H
> +#define __CAMERA_ISP_H
> +
> +#include <linux/cdev.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/ioctl.h>
> +#include <linux/irqreturn.h>
> +#include <linux/miscdevice.h>
> +#include <linux/pm_qos.h>
> +#include <linux/scatterlist.h>
> +
> +#include "mtk_cam-scp.h"
> +#include "mtk_cam-v4l2-util.h"
> +
> +#define CAM_A_MAX_WIDTH		3328
> +#define CAM_A_MAX_HEIGHT		2496
> +#define CAM_B_MAX_WIDTH		5376
> +#define CAM_B_MAX_HEIGHT		4032
> +
> +#define CAM_MIN_WIDTH			80
> +#define CAM_MIN_HEIGHT			60
> +
> +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> +
> +#define R_IMGO				BIT(0)
> +#define R_RRZO				BIT(1)
> +#define R_AAO				BIT(3)
> +#define R_AFO				BIT(4)
> +#define R_LCSO				BIT(5)
> +#define R_PDO				BIT(6)
> +#define R_LMVO				BIT(7)
> +#define R_FLKO				BIT(8)
> +#define R_RSSO				BIT(9)
> +#define R_PSO				BIT(10)
> +
> +#define CQ_BUFFER_COUNT		3
> +#define IRQ_DATA_BUF_SIZE		4
> +#define CQ_ADDRESS_OFFSET		0x640
> +
> +#define ISP_COMPOSING_MAX_NUM		4
> +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> +
> +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> +			"fbc1(0x%x), fbc2(0x%x)\n"
> +
> +/*
> + * In order with the sequence of device nodes defined in dtsi rule,
> + * one hardware module should be mapping to one node.
> + */
> +enum isp_dev_node_enum {
> +	ISP_CAMSYS_CONFIG_IDX = 0,
> +	ISP_CAM_UNI_IDX,
> +	ISP_CAM_A_IDX,
> +	ISP_CAM_B_IDX,
> +	ISP_DEV_NODE_NUM
> +};
> +
> +/* Image RAW path for ISP P1 module. */
> +enum isp_raw_path_enum {
> +	ISP_PROCESS_RAW_PATH = 0,
> +	ISP_PURE_RAW_PATH
> +};
> +
> +/* State for struct mtk_isp_p1_ctx: composer_state */
> +enum  {
> +	SCP_ON = 0,
> +	SCP_OFF
> +};

Hmm, looks like bool.

> +
> +enum {
> +	IMG_FMT_UNKNOWN		= 0x0000,
> +	IMG_FMT_RAW_START	= 0x2200,
> +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> +	IMG_FMT_BAYER10,
> +	IMG_FMT_BAYER12,
> +	IMG_FMT_BAYER14,
> +	IMG_FMT_FG_BAYER8,
> +	IMG_FMT_FG_BAYER10,
> +	IMG_FMT_FG_BAYER12,
> +	IMG_FMT_FG_BAYER14,
> +};
> +
> +enum {
> +	RAW_PXL_ID_B = 0,
> +	RAW_PXL_ID_GB,
> +	RAW_PXL_ID_GR,
> +	RAW_PXL_ID_R
> +};

Please use macros with explicitly assigned values for hardware/firmware ABI
definitions.

> +
> +struct isp_queue {
> +	struct list_head queue;
> +	atomic_t queue_cnt;
> +	spinlock_t lock; /* queue attributes protection */
> +};
> +
> +struct isp_thread {
> +	struct task_struct *thread;
> +	wait_queue_head_t wq;
> +};
> +
> +struct mtk_isp_queue_work {
> +	union {
> +		struct mtk_isp_scp_p1_cmd cmd;
> +		struct p1_frame_param frameparams;
> +	};
> +	struct list_head list_entry;
> +	enum mtk_isp_scp_type type;
> +};
> +
> +struct mtk_cam_dev_stat_event_data {
> +	__u32 frame_seq_no;
> +	__u32 meta0_vb2_index;
> +	__u32 meta1_vb2_index;
> +	__u32 irq_status_mask;
> +	__u32 dma_status_mask;
> +};
> +
> +struct mtk_isp_queue_job {
> +	struct list_head list_entry;
> +	struct list_head list_buf;
> +	unsigned int request_fd;
> +	unsigned int frame_seq_no;
> +};

Please document the structs above using kerneldoc.

> +
> +/*
> + * struct isp_device - the ISP device information
> + *
> + * @dev: Pointer to struct device
> + * @regs: Camera ISP base register address
> + * @spinlock_irq: Used to protect register read/write data
> + * @current_frame: Current frame sequence number, set when SOF
> + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> + * @sof_count: The accumulated SOF counter
> + * @isp_hw_module: Identity camera A or B
> + *
> + */
> +struct isp_device {

mtk_isp_device?

> +	struct device *dev;
> +	void __iomem *regs;
> +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> +	unsigned int current_frame;
> +	unsigned int meta0_vb2_index;
> +	unsigned int meta1_vb2_index;
> +	u8 sof_count;
> +	u8 isp_hw_module;
> +};
> +
> +/*
> + * struct mtk_isp_p1_ctx - the ISP device information
> + *
> + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> + * @composer_tx_thread: TX Thread for SCP data tranmission
> + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> + * @ipi_occupied: The total number of SCP TX data has beent sent
> + * @scp_state: The state of SCP control
> + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> + * @composed_frame_id: The ack. frame sequence by SCP
> + * @composer_deinit_thread: The de-initialized thread
> + * @p1_enqueue_list: Queue for ISP frame buffers
> + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> + * @irq_data_start: Start index of irq_event_datas ring buffer
> + * @irq_data_end: End index of irq_event_datas ring buffer
> + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> + * @scp_mem_pa: DMA address for SCP device
> + * @scp_mem_iova: DMA address for ISP HW DMA devices
> + * @frame_seq_no: Sequence number for ISP frame buffer
> + * @isp_hw_module: Active camera HW module
> + * @num_clks: The number of driver's clock
> + * @clk_list: The list of clock data
> + * @lock: Lock to protect context operations
> + *
> + */
> +struct mtk_isp_p1_ctx {
> +	struct isp_queue composer_txlist;
> +	struct isp_thread composer_tx_thread;
> +	atomic_t cmd_queued;
> +	atomic_t ipi_occupied;
> +	atomic_t scp_state;
> +	atomic_t composing_frame;
> +	atomic_t composed_frame_id;
> +	struct isp_thread composer_deinit_thread;
> +	struct isp_queue p1_enqueue_list;
> +	struct isp_thread isp_deque_thread;
> +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> +	atomic_t irq_data_start;
> +	atomic_t irq_data_end;
> +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */

Already documented in kerneldoc.

> +	dma_addr_t scp_mem_pa;
> +	dma_addr_t scp_mem_iova;
> +	int frame_seq_no;
> +	unsigned int isp_hw_module;
> +	unsigned int isp_raw_path;

Not documented above.

> +	unsigned int num_clks;
> +	struct clk_bulk_data *clk_list;
> +	struct mutex lock; /* Protect context operations */

Already documented in kerneldoc.

> +};
> +
> +struct isp_p1_device {
> +	struct platform_device *pdev;
> +	struct platform_device *scp_pdev;
> +	struct rproc *rproc_handle;
> +	struct mtk_isp_p1_ctx isp_ctx;
> +	struct mtk_cam_dev cam_dev;
> +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> +};

Please document in a kerneldoc comment.

> +
> +static inline struct isp_p1_device *
> +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> +{
> +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> +}
> +
> +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> +{
> +	return ((struct isp_p1_device *)dev_get_drvdata(dev));

No need to cast. And, I don't think we need a function for this, just call
dev_get_drvdata() directly.

Best regards,
Tomasz


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
  2019-06-11  3:53     ` Jungo Lin
  (?)
@ 2019-07-10  9:58         ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:58 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	sean.cheng-NuS5LvNUpcJWk0Htik3J/w,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	rynn.wu-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	robh-DgEjT+Ai2ygdnm+yROfE0A, ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	hverkuil-qWit8jRvyhVmR6Xm/wNWPw,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-media-u79uwXL29TY76Z2rM5mHXA

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> This patch adds communication with the co-processor on the SoC
> through the SCP driver. It supports bi-directional commands
> to exchange data and perform command flow control function.
> 
> Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
>  3 files changed, 579 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> new file mode 100644
> index 000000000000..04519d0b942f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> @@ -0,0 +1,371 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/kthread.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +
> +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);

Is there any point to set them if we are deinitalizing? Moreover,
isp_composer_init() would set them once we start again.

> +
> +	spin_lock(&isp_ctx->composer_txlist.lock);
> +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> +				 &isp_ctx->composer_txlist.queue,
> +				 list_entry) {
> +		list_del(&ipi_job->list_entry);
> +		kfree(ipi_job);
> +	}
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (isp_ctx->composer_tx_thread.thread) {
> +		kthread_stop(isp_ctx->composer_tx_thread.thread);

Shouldn't the thread be stopped at this point already? If not, wouldn't the
atomic_set() at the beginning of this function confuse it?

In any case, this should be greatly simplified after we move to a workqueue,
with one work per one task to do, as per other comments.

> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		isp_ctx->composer_tx_thread.thread = NULL;
> +	}
> +
> +	if (isp_ctx->composer_deinit_thread.thread) {
> +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread = NULL;
> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);

No need to use the sync variant.

> +}
> +
> +/*
> + * Two kinds of flow control in isp_composer_tx_work.
> + *
> + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> + * It is controlled by ipi_occupied.

ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?

> + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.

What does it mean and why is it so?

> + *
> + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> + * It is controlled by composing_frame.
> + * Frame buffer is sent by SCP_ISP_FRAME command.

Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
and case 2?

> + */
> +static int isp_composer_tx_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> +	int ret;
> +
> +	while (1) {
> +		ret = wait_event_interruptible
> +			(isp_ctx->composer_tx_thread.wq,
> +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM &&
> +			 atomic_read(&isp_ctx->composing_frame)
> +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM) ||
> +			 kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock(&composer_txlist->lock);
> +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> +						 &composer_txlist->queue,
> +						 list_entry) {
> +				if (isp_composer_work->type == SCP_ISP_CMD) {
> +					dev_dbg(dev, "Found a cmd\n");
> +					break;
> +				}
> +			}
> +		} else {
> +			if (atomic_read(&isp_ctx->composing_frame) >=
> +				ISP_FRAME_COMPOSING_MAX_NUM) {
> +				spin_unlock(&composer_txlist->lock);
> +				continue;
> +			}
> +			isp_composer_work =
> +			    list_first_entry_or_null
> +				(&composer_txlist->queue,
> +				 struct mtk_isp_queue_work,
> +				 list_entry);
> +		}

I don't understand why this special handling of CMD vs FRAME is here, so I
might be missing something, but would we really lose anything if we just
simply removed it and queued everything in order?

Moreover, in V4L2, buffer queue and control operations are serialized wrt
each other, so we probably wouldn't even have a chance to hit a case when we
need to prioritize a CMD IPI over a FRAME IPI.

> +
> +		list_del(&isp_composer_work->list_entry);
> +		atomic_dec(&composer_txlist->queue_cnt);
> +		spin_unlock(&composer_txlist->lock);
> +
> +		if (isp_composer_work->type == SCP_ISP_CMD) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_CMD,
> +				 &isp_composer_work->cmd,
> +				 sizeof(isp_composer_work->cmd),
> +				 0);
> +			atomic_dec(&isp_ctx->cmd_queued);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			dev_dbg(dev,
> +				"%s cmd id %d sent, %d ipi buf occupied",
> +				__func__,
> +				isp_composer_work->cmd.cmd_id,
> +				atomic_read(&isp_ctx->ipi_occupied));
> +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_FRAME,
> +				 &isp_composer_work->frameparams,
> +				 sizeof(isp_composer_work->frameparams),
> +				 0);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			atomic_inc(&isp_ctx->composing_frame);

Why do we need composing frame here, if ipi_occupied already limits us to 3?

> +			dev_dbg(dev,
> +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> +				__func__,
> +				isp_composer_work->frameparams.frame_seq_no,
> +				atomic_read(&isp_ctx->ipi_occupied),
> +				atomic_read(&isp_ctx->composing_frame));
> +		} else {
> +			dev_err(dev,
> +				"ignore IPI type: %d!\n",
> +				isp_composer_work->type);
> +		}
> +		kfree(isp_composer_work);
> +	}
> +	return ret;
> +}

The function above is way too complicated than it should be. I'd suggest a
model similar to what we ended up in the DIP driver:
>  - a freezable workqueue created for ISP composing works,
>  - each ISP composing work entry would have a struct work_struct embedded,
>  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
>    above,
>  - the workqueue would keep a queue of works itself, so driver's own list
>    wouldn't be needed anymore,
>  - similarly, each execution of the work func would operate on its own ISP
>    composing work, so things like checking for list emptiness, waiting for
>    work to be queued, etc. wouldn't be needed,
>  - freezability of the workqueue would ensure nice synchonization with
>    system suspend/resume (although one would still need to wait for the
>    hardware/firmware to complete).

WDYT?

> +
> +static int isp_composer_deinit_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> +				 kthread_should_stop());
> +
> +	dev_dbg(dev, "%s run deinit", __func__);
> +	isp_composer_deinit(isp_ctx);
> +
> +	return 0;
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
contain some extra bytes at the end.)

> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> +		return;
> +
> +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> +		dev_dbg(dev, "ack frame_num:%d",
> +			ipi_msg->ack_info.frame_seq_no);
> +		atomic_set(&isp_ctx->composed_frame_id,
> +			   ipi_msg->ack_info.frame_seq_no);

I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
frames and we should handle that somehow.

> +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> +	}
> +
> +	atomic_dec_return(&isp_ctx->ipi_occupied);
> +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +}
> +
> +int isp_composer_init(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp_pdev,
> +			       SCP_IPI_ISP_CMD,
> +			       isp_composer_handler,
> +			       isp_ctx);
> +	if (ret)
> +		return ret;
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);
> +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (!isp_ctx->composer_tx_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> +		isp_ctx->composer_tx_thread.thread =
> +			kthread_run(isp_composer_tx_work, isp_ctx,
> +				    "isp_composer_tx");
> +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_tx_thread.thread = NULL;
> +			goto nomem;

Why nomem?

> +		}
> +	} else {
> +		dev_warn(dev, "old tx thread is existed\n");

This shouldn't be possible to happen.

> +	}
> +
> +	if (!isp_ctx->composer_deinit_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread =
> +			kthread_run(isp_composer_deinit_work, isp_ctx,
> +				    "isp_composer_deinit_work");

Why do we need to deinit from another kthread?

> +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_deinit_thread.thread = NULL;
> +			goto nomem;
> +		}
> +	} else {
> +		dev_warn(dev, "old rx thread is existed\n");

rx? The code above seems to refer to deinit.

> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return 0;
> +
> +nomem:
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return -ENOMEM;

We should return the original error code here.

> +}
> +
> +void isp_composer_enqueue(struct device *dev,
> +			  void *data,
> +			  enum mtk_isp_scp_type type)
> +{
> +	struct mtk_isp_queue_work *isp_composer_work;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);

For most of the cases, it should be possible to preallocate this, e.g.
>  - for FRAME, this could be inside the request struct,
>  - for buffer queue it could be inside the buffer struct.

I'd suggest making the caller responsible for allocating if needed.

> +	isp_composer_work->type = type;
> +
> +	switch (type) {
> +	case SCP_ISP_CMD:
> +		memcpy(&isp_composer_work->cmd, data,
> +		       sizeof(isp_composer_work->cmd));
> +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> +			isp_composer_work->cmd.cmd_id);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		atomic_inc(&isp_ctx->cmd_queued);
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		break;
> +	case SCP_ISP_FRAME:
> +		memcpy(&isp_composer_work->frameparams, data,
> +		       sizeof(isp_composer_work->frameparams));
> +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> +			isp_composer_work->frameparams.frame_seq_no);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);

The code in both cases is almost exactly the same. The only difference is
the memcpy destination and size and whether isp_ctx->cmd_queued is
incremented or not.

The memcpy will go away if my comment above is addressed and so that would
go down to making the cmd_queued increment conditional.

> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
> +void isp_composer_hw_init(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;

Should we also specify the size of the buffer? Otherwise we could end up
with some undetectable overruns.

> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_meta_config(struct device *dev,
> +			      unsigned int dma)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_config(struct device *dev,
> +			    struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_stream(struct device *dev, int on)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_deinit(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +
> +	/* Wait for ISP_CMD_DEINIT command is handled done */
> +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> +				 msecs_to_jiffies(2000));
> +	if (ret)
> +		return;
> +
> +	dev_warn(dev, "Timeout & local de-init\n");
> +	isp_composer_deinit(isp_ctx);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> new file mode 100644
> index 000000000000..fbd8593e9c2d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> @@ -0,0 +1,207 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _MTK_ISP_SCP_H
> +#define _MTK_ISP_SCP_H
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam-v4l2-util.h"
> +
> +/*
> + * struct img_size - image size information.
> + *
> + * @w: image width, the unit is pixel
> + * @h: image height, the unit is pixel
> + * @xsize: bytes per line based on width.
> + * @stride: bytes per line when changing line.
> + *          Normally, calculate new STRIDE based on
> + *          xsize + HW constrain(page or align).
> + *
> + */
> +struct img_size {
> +	__u32 w;
> +	__u32 h;
> +	__u32 xsize;
> +	__u32 stride;
> +} __packed;
> +
> +/*
> + * struct img_buffer - buffer address information.
> + *
> + * @iova: DMA address for external devices.
> + * @scp_addr: SCP address for external co-process unit.
> + *
> + */
> +struct img_buffer {
> +	__u32 iova;
> +	__u32 scp_addr;
> +} __packed;
> +
> +struct p1_img_crop {
> +	__u32 left;
> +	__u32 top;
> +	__u32 width;
> +	__u32 height;
> +} __packed;
> +
> +struct p1_img_output {
> +	struct img_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	__u8 pixel_byte;
> +	__u32 img_fmt;
> +} __packed;

Please document.

> +
> +/*
> + * struct cfg_in_param - image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: indicate the sensor mode.
> + *              1: continuous
> + *              0: single
> + * @subsample: indicate to enables SOF subsample or not.
> + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: describe input data pattern.
> + * @raw_pixel_id: bayer sequence.
> + * @tg_fps: the fps rate of TG (time generator).
> + * @img_fmt: the image format of input source.
> + * @p1_img_crop: the crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	__u8 continuous;
> +	__u8 subsample;
> +	__u8 pixel_mode;
> +	__u8 data_pattern;
> +	__u8 raw_pixel_id;
> +	__u16 tg_fps;
> +	__u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - the image output parameters of main stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @pure_raw: indicate the image path control.
> + *            1: pure raw
> + *            0: processing raw
> + * @pure_raw_pack: indicate the image is packed or not.
> + *                 1: packed mode
> + *                 0: unpacked mode
> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	/* Bypass main out parameters */
> +	__u8 bypass;
> +	/* Control HW image raw path */
> +	__u8 pure_raw;
> +	/* Control HW image pack function */

No need for these inline comments.

> +	__u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - the image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	/* Bypass resize parameters */

No need for this inline comment.

> +	__u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_meta_out_param - output meta information.
> + *
> + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> + *
> + */
> +struct cfg_meta_out_param {
> +	__u32 enabled_meta_dmas;
> +} __packed;
> +
> +struct p1_config_param {
> +	/* Sensor/TG info */
> +	struct cfg_in_param cfg_in_param;
> +	/* IMGO DMA */
> +	struct cfg_main_out_param cfg_main_param;
> +	/* RRZO DMA */
> +	struct cfg_resize_out_param cfg_resize_param;
> +	/* 3A DMAs and other. */
> +	struct cfg_meta_out_param cfg_meta_param;

Please change the inline comments to a kerneldoc comment at the top.

> +} __packed;
> +
> +struct p1_frame_param {
> +	/* frame sequence number */
> +	__u32 frame_seq_no;
> +	/* SOF index */
> +	__u32 sof_idx;
> +	/* The memory address of tuning buffer from user space */

Ditto.

> +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +struct P1_meta_frame {
> +	__u32 enabled_dma;
> +	__u32 vb_index;
> +	struct img_buffer meta_addr;
> +} __packed;
> +
> +struct isp_init_info {
> +	__u8 hw_module;
> +	struct img_buffer cq_addr;
> +} __packed;
> +
> +struct isp_ack_info {
> +	__u8 cmd_id;
> +	__u32 frame_seq_no;
> +} __packed;
> +
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_CONFIG_META,
> +	ISP_CMD_ENQUEUE_META,
> +	ISP_CMD_RESERVED,
> +};
> +
> +struct mtk_isp_scp_p1_cmd {
> +	__u8 cmd_id;
> +	union {
> +		struct isp_init_info frameparam;
> +		struct p1_config_param config_param;
> +		struct cfg_meta_out_param cfg_meta_out_param;
> +		struct P1_meta_frame meta_frame;
> +		__u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +enum mtk_isp_scp_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};

Please document all the structs and enum above using kerneldoc.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-10  9:58         ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:58 UTC (permalink / raw)
  To: Jungo Lin
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> This patch adds communication with the co-processor on the SoC
> through the SCP driver. It supports bi-directional commands
> to exchange data and perform command flow control function.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
>  3 files changed, 579 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> new file mode 100644
> index 000000000000..04519d0b942f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> @@ -0,0 +1,371 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/kthread.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +
> +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);

Is there any point to set them if we are deinitalizing? Moreover,
isp_composer_init() would set them once we start again.

> +
> +	spin_lock(&isp_ctx->composer_txlist.lock);
> +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> +				 &isp_ctx->composer_txlist.queue,
> +				 list_entry) {
> +		list_del(&ipi_job->list_entry);
> +		kfree(ipi_job);
> +	}
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (isp_ctx->composer_tx_thread.thread) {
> +		kthread_stop(isp_ctx->composer_tx_thread.thread);

Shouldn't the thread be stopped at this point already? If not, wouldn't the
atomic_set() at the beginning of this function confuse it?

In any case, this should be greatly simplified after we move to a workqueue,
with one work per one task to do, as per other comments.

> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		isp_ctx->composer_tx_thread.thread = NULL;
> +	}
> +
> +	if (isp_ctx->composer_deinit_thread.thread) {
> +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread = NULL;
> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);

No need to use the sync variant.

> +}
> +
> +/*
> + * Two kinds of flow control in isp_composer_tx_work.
> + *
> + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> + * It is controlled by ipi_occupied.

ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?

> + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.

What does it mean and why is it so?

> + *
> + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> + * It is controlled by composing_frame.
> + * Frame buffer is sent by SCP_ISP_FRAME command.

Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
and case 2?

> + */
> +static int isp_composer_tx_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> +	int ret;
> +
> +	while (1) {
> +		ret = wait_event_interruptible
> +			(isp_ctx->composer_tx_thread.wq,
> +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM &&
> +			 atomic_read(&isp_ctx->composing_frame)
> +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM) ||
> +			 kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock(&composer_txlist->lock);
> +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> +						 &composer_txlist->queue,
> +						 list_entry) {
> +				if (isp_composer_work->type == SCP_ISP_CMD) {
> +					dev_dbg(dev, "Found a cmd\n");
> +					break;
> +				}
> +			}
> +		} else {
> +			if (atomic_read(&isp_ctx->composing_frame) >=
> +				ISP_FRAME_COMPOSING_MAX_NUM) {
> +				spin_unlock(&composer_txlist->lock);
> +				continue;
> +			}
> +			isp_composer_work =
> +			    list_first_entry_or_null
> +				(&composer_txlist->queue,
> +				 struct mtk_isp_queue_work,
> +				 list_entry);
> +		}

I don't understand why this special handling of CMD vs FRAME is here, so I
might be missing something, but would we really lose anything if we just
simply removed it and queued everything in order?

Moreover, in V4L2, buffer queue and control operations are serialized wrt
each other, so we probably wouldn't even have a chance to hit a case when we
need to prioritize a CMD IPI over a FRAME IPI.

> +
> +		list_del(&isp_composer_work->list_entry);
> +		atomic_dec(&composer_txlist->queue_cnt);
> +		spin_unlock(&composer_txlist->lock);
> +
> +		if (isp_composer_work->type == SCP_ISP_CMD) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_CMD,
> +				 &isp_composer_work->cmd,
> +				 sizeof(isp_composer_work->cmd),
> +				 0);
> +			atomic_dec(&isp_ctx->cmd_queued);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			dev_dbg(dev,
> +				"%s cmd id %d sent, %d ipi buf occupied",
> +				__func__,
> +				isp_composer_work->cmd.cmd_id,
> +				atomic_read(&isp_ctx->ipi_occupied));
> +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_FRAME,
> +				 &isp_composer_work->frameparams,
> +				 sizeof(isp_composer_work->frameparams),
> +				 0);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			atomic_inc(&isp_ctx->composing_frame);

Why do we need composing frame here, if ipi_occupied already limits us to 3?

> +			dev_dbg(dev,
> +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> +				__func__,
> +				isp_composer_work->frameparams.frame_seq_no,
> +				atomic_read(&isp_ctx->ipi_occupied),
> +				atomic_read(&isp_ctx->composing_frame));
> +		} else {
> +			dev_err(dev,
> +				"ignore IPI type: %d!\n",
> +				isp_composer_work->type);
> +		}
> +		kfree(isp_composer_work);
> +	}
> +	return ret;
> +}

The function above is way too complicated than it should be. I'd suggest a
model similar to what we ended up in the DIP driver:
>  - a freezable workqueue created for ISP composing works,
>  - each ISP composing work entry would have a struct work_struct embedded,
>  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
>    above,
>  - the workqueue would keep a queue of works itself, so driver's own list
>    wouldn't be needed anymore,
>  - similarly, each execution of the work func would operate on its own ISP
>    composing work, so things like checking for list emptiness, waiting for
>    work to be queued, etc. wouldn't be needed,
>  - freezability of the workqueue would ensure nice synchonization with
>    system suspend/resume (although one would still need to wait for the
>    hardware/firmware to complete).

WDYT?

> +
> +static int isp_composer_deinit_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> +				 kthread_should_stop());
> +
> +	dev_dbg(dev, "%s run deinit", __func__);
> +	isp_composer_deinit(isp_ctx);
> +
> +	return 0;
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
contain some extra bytes at the end.)

> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> +		return;
> +
> +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> +		dev_dbg(dev, "ack frame_num:%d",
> +			ipi_msg->ack_info.frame_seq_no);
> +		atomic_set(&isp_ctx->composed_frame_id,
> +			   ipi_msg->ack_info.frame_seq_no);

I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
frames and we should handle that somehow.

> +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> +	}
> +
> +	atomic_dec_return(&isp_ctx->ipi_occupied);
> +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +}
> +
> +int isp_composer_init(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp_pdev,
> +			       SCP_IPI_ISP_CMD,
> +			       isp_composer_handler,
> +			       isp_ctx);
> +	if (ret)
> +		return ret;
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);
> +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (!isp_ctx->composer_tx_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> +		isp_ctx->composer_tx_thread.thread =
> +			kthread_run(isp_composer_tx_work, isp_ctx,
> +				    "isp_composer_tx");
> +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_tx_thread.thread = NULL;
> +			goto nomem;

Why nomem?

> +		}
> +	} else {
> +		dev_warn(dev, "old tx thread is existed\n");

This shouldn't be possible to happen.

> +	}
> +
> +	if (!isp_ctx->composer_deinit_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread =
> +			kthread_run(isp_composer_deinit_work, isp_ctx,
> +				    "isp_composer_deinit_work");

Why do we need to deinit from another kthread?

> +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_deinit_thread.thread = NULL;
> +			goto nomem;
> +		}
> +	} else {
> +		dev_warn(dev, "old rx thread is existed\n");

rx? The code above seems to refer to deinit.

> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return 0;
> +
> +nomem:
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return -ENOMEM;

We should return the original error code here.

> +}
> +
> +void isp_composer_enqueue(struct device *dev,
> +			  void *data,
> +			  enum mtk_isp_scp_type type)
> +{
> +	struct mtk_isp_queue_work *isp_composer_work;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);

For most of the cases, it should be possible to preallocate this, e.g.
>  - for FRAME, this could be inside the request struct,
>  - for buffer queue it could be inside the buffer struct.

I'd suggest making the caller responsible for allocating if needed.

> +	isp_composer_work->type = type;
> +
> +	switch (type) {
> +	case SCP_ISP_CMD:
> +		memcpy(&isp_composer_work->cmd, data,
> +		       sizeof(isp_composer_work->cmd));
> +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> +			isp_composer_work->cmd.cmd_id);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		atomic_inc(&isp_ctx->cmd_queued);
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		break;
> +	case SCP_ISP_FRAME:
> +		memcpy(&isp_composer_work->frameparams, data,
> +		       sizeof(isp_composer_work->frameparams));
> +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> +			isp_composer_work->frameparams.frame_seq_no);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);

The code in both cases is almost exactly the same. The only difference is
the memcpy destination and size and whether isp_ctx->cmd_queued is
incremented or not.

The memcpy will go away if my comment above is addressed and so that would
go down to making the cmd_queued increment conditional.

> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
> +void isp_composer_hw_init(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;

Should we also specify the size of the buffer? Otherwise we could end up
with some undetectable overruns.

> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_meta_config(struct device *dev,
> +			      unsigned int dma)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_config(struct device *dev,
> +			    struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_stream(struct device *dev, int on)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_deinit(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +
> +	/* Wait for ISP_CMD_DEINIT command is handled done */
> +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> +				 msecs_to_jiffies(2000));
> +	if (ret)
> +		return;
> +
> +	dev_warn(dev, "Timeout & local de-init\n");
> +	isp_composer_deinit(isp_ctx);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> new file mode 100644
> index 000000000000..fbd8593e9c2d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> @@ -0,0 +1,207 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _MTK_ISP_SCP_H
> +#define _MTK_ISP_SCP_H
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam-v4l2-util.h"
> +
> +/*
> + * struct img_size - image size information.
> + *
> + * @w: image width, the unit is pixel
> + * @h: image height, the unit is pixel
> + * @xsize: bytes per line based on width.
> + * @stride: bytes per line when changing line.
> + *          Normally, calculate new STRIDE based on
> + *          xsize + HW constrain(page or align).
> + *
> + */
> +struct img_size {
> +	__u32 w;
> +	__u32 h;
> +	__u32 xsize;
> +	__u32 stride;
> +} __packed;
> +
> +/*
> + * struct img_buffer - buffer address information.
> + *
> + * @iova: DMA address for external devices.
> + * @scp_addr: SCP address for external co-process unit.
> + *
> + */
> +struct img_buffer {
> +	__u32 iova;
> +	__u32 scp_addr;
> +} __packed;
> +
> +struct p1_img_crop {
> +	__u32 left;
> +	__u32 top;
> +	__u32 width;
> +	__u32 height;
> +} __packed;
> +
> +struct p1_img_output {
> +	struct img_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	__u8 pixel_byte;
> +	__u32 img_fmt;
> +} __packed;

Please document.

> +
> +/*
> + * struct cfg_in_param - image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: indicate the sensor mode.
> + *              1: continuous
> + *              0: single
> + * @subsample: indicate to enables SOF subsample or not.
> + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: describe input data pattern.
> + * @raw_pixel_id: bayer sequence.
> + * @tg_fps: the fps rate of TG (time generator).
> + * @img_fmt: the image format of input source.
> + * @p1_img_crop: the crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	__u8 continuous;
> +	__u8 subsample;
> +	__u8 pixel_mode;
> +	__u8 data_pattern;
> +	__u8 raw_pixel_id;
> +	__u16 tg_fps;
> +	__u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - the image output parameters of main stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @pure_raw: indicate the image path control.
> + *            1: pure raw
> + *            0: processing raw
> + * @pure_raw_pack: indicate the image is packed or not.
> + *                 1: packed mode
> + *                 0: unpacked mode
> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	/* Bypass main out parameters */
> +	__u8 bypass;
> +	/* Control HW image raw path */
> +	__u8 pure_raw;
> +	/* Control HW image pack function */

No need for these inline comments.

> +	__u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - the image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	/* Bypass resize parameters */

No need for this inline comment.

> +	__u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_meta_out_param - output meta information.
> + *
> + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> + *
> + */
> +struct cfg_meta_out_param {
> +	__u32 enabled_meta_dmas;
> +} __packed;
> +
> +struct p1_config_param {
> +	/* Sensor/TG info */
> +	struct cfg_in_param cfg_in_param;
> +	/* IMGO DMA */
> +	struct cfg_main_out_param cfg_main_param;
> +	/* RRZO DMA */
> +	struct cfg_resize_out_param cfg_resize_param;
> +	/* 3A DMAs and other. */
> +	struct cfg_meta_out_param cfg_meta_param;

Please change the inline comments to a kerneldoc comment at the top.

> +} __packed;
> +
> +struct p1_frame_param {
> +	/* frame sequence number */
> +	__u32 frame_seq_no;
> +	/* SOF index */
> +	__u32 sof_idx;
> +	/* The memory address of tuning buffer from user space */

Ditto.

> +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +struct P1_meta_frame {
> +	__u32 enabled_dma;
> +	__u32 vb_index;
> +	struct img_buffer meta_addr;
> +} __packed;
> +
> +struct isp_init_info {
> +	__u8 hw_module;
> +	struct img_buffer cq_addr;
> +} __packed;
> +
> +struct isp_ack_info {
> +	__u8 cmd_id;
> +	__u32 frame_seq_no;
> +} __packed;
> +
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_CONFIG_META,
> +	ISP_CMD_ENQUEUE_META,
> +	ISP_CMD_RESERVED,
> +};
> +
> +struct mtk_isp_scp_p1_cmd {
> +	__u8 cmd_id;
> +	union {
> +		struct isp_init_info frameparam;
> +		struct p1_config_param config_param;
> +		struct cfg_meta_out_param cfg_meta_out_param;
> +		struct P1_meta_frame meta_frame;
> +		__u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +enum mtk_isp_scp_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};

Please document all the structs and enum above using kerneldoc.

Best regards,
Tomasz


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-10  9:58         ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-10  9:58 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Jungo,

On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> This patch adds communication with the co-processor on the SoC
> through the SCP driver. It supports bi-directional commands
> to exchange data and perform command flow control function.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> This patch depends on "Add support for mt8183 SCP"[1].
> 
> [1] https://patchwork.kernel.org/cover/10972143/
> ---
>  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
>  3 files changed, 579 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> 

Thanks for the patch! Please see my comments inline.

[snip]

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> new file mode 100644
> index 000000000000..04519d0b942f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> @@ -0,0 +1,371 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2018 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/kthread.h>
> +#include <linux/platform_data/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include "mtk_cam.h"
> +
> +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> +{
> +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);

Is there any point to set them if we are deinitalizing? Moreover,
isp_composer_init() would set them once we start again.

> +
> +	spin_lock(&isp_ctx->composer_txlist.lock);
> +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> +				 &isp_ctx->composer_txlist.queue,
> +				 list_entry) {
> +		list_del(&ipi_job->list_entry);
> +		kfree(ipi_job);
> +	}
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (isp_ctx->composer_tx_thread.thread) {
> +		kthread_stop(isp_ctx->composer_tx_thread.thread);

Shouldn't the thread be stopped at this point already? If not, wouldn't the
atomic_set() at the beginning of this function confuse it?

In any case, this should be greatly simplified after we move to a workqueue,
with one work per one task to do, as per other comments.

> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		isp_ctx->composer_tx_thread.thread = NULL;
> +	}
> +
> +	if (isp_ctx->composer_deinit_thread.thread) {
> +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread = NULL;
> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	pm_runtime_put_sync(&p1_dev->pdev->dev);

No need to use the sync variant.

> +}
> +
> +/*
> + * Two kinds of flow control in isp_composer_tx_work.
> + *
> + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> + * It is controlled by ipi_occupied.

ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?

> + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.

What does it mean and why is it so?

> + *
> + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> + * It is controlled by composing_frame.
> + * Frame buffer is sent by SCP_ISP_FRAME command.

Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
and case 2?

> + */
> +static int isp_composer_tx_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> +	int ret;
> +
> +	while (1) {
> +		ret = wait_event_interruptible
> +			(isp_ctx->composer_tx_thread.wq,
> +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM &&
> +			 atomic_read(&isp_ctx->composing_frame)
> +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> +			 atomic_read(&isp_ctx->ipi_occupied)
> +				< ISP_COMPOSING_MAX_NUM) ||
> +			 kthread_should_stop());
> +
> +		if (kthread_should_stop())
> +			break;
> +
> +		spin_lock(&composer_txlist->lock);
> +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> +						 &composer_txlist->queue,
> +						 list_entry) {
> +				if (isp_composer_work->type == SCP_ISP_CMD) {
> +					dev_dbg(dev, "Found a cmd\n");
> +					break;
> +				}
> +			}
> +		} else {
> +			if (atomic_read(&isp_ctx->composing_frame) >=
> +				ISP_FRAME_COMPOSING_MAX_NUM) {
> +				spin_unlock(&composer_txlist->lock);
> +				continue;
> +			}
> +			isp_composer_work =
> +			    list_first_entry_or_null
> +				(&composer_txlist->queue,
> +				 struct mtk_isp_queue_work,
> +				 list_entry);
> +		}

I don't understand why this special handling of CMD vs FRAME is here, so I
might be missing something, but would we really lose anything if we just
simply removed it and queued everything in order?

Moreover, in V4L2, buffer queue and control operations are serialized wrt
each other, so we probably wouldn't even have a chance to hit a case when we
need to prioritize a CMD IPI over a FRAME IPI.

> +
> +		list_del(&isp_composer_work->list_entry);
> +		atomic_dec(&composer_txlist->queue_cnt);
> +		spin_unlock(&composer_txlist->lock);
> +
> +		if (isp_composer_work->type == SCP_ISP_CMD) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_CMD,
> +				 &isp_composer_work->cmd,
> +				 sizeof(isp_composer_work->cmd),
> +				 0);
> +			atomic_dec(&isp_ctx->cmd_queued);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			dev_dbg(dev,
> +				"%s cmd id %d sent, %d ipi buf occupied",
> +				__func__,
> +				isp_composer_work->cmd.cmd_id,
> +				atomic_read(&isp_ctx->ipi_occupied));
> +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> +			scp_ipi_send
> +				(p1_dev->scp_pdev,
> +				 SCP_IPI_ISP_FRAME,
> +				 &isp_composer_work->frameparams,
> +				 sizeof(isp_composer_work->frameparams),
> +				 0);
> +			atomic_inc(&isp_ctx->ipi_occupied);
> +			atomic_inc(&isp_ctx->composing_frame);

Why do we need composing frame here, if ipi_occupied already limits us to 3?

> +			dev_dbg(dev,
> +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> +				__func__,
> +				isp_composer_work->frameparams.frame_seq_no,
> +				atomic_read(&isp_ctx->ipi_occupied),
> +				atomic_read(&isp_ctx->composing_frame));
> +		} else {
> +			dev_err(dev,
> +				"ignore IPI type: %d!\n",
> +				isp_composer_work->type);
> +		}
> +		kfree(isp_composer_work);
> +	}
> +	return ret;
> +}

The function above is way too complicated than it should be. I'd suggest a
model similar to what we ended up in the DIP driver:
>  - a freezable workqueue created for ISP composing works,
>  - each ISP composing work entry would have a struct work_struct embedded,
>  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
>    above,
>  - the workqueue would keep a queue of works itself, so driver's own list
>    wouldn't be needed anymore,
>  - similarly, each execution of the work func would operate on its own ISP
>    composing work, so things like checking for list emptiness, waiting for
>    work to be queued, etc. wouldn't be needed,
>  - freezability of the workqueue would ensure nice synchonization with
>    system suspend/resume (although one would still need to wait for the
>    hardware/firmware to complete).

WDYT?

> +
> +static int isp_composer_deinit_work(void *data)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> +	struct device *dev = &p1_dev->pdev->dev;
> +
> +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> +				 kthread_should_stop());
> +
> +	dev_dbg(dev, "%s run deinit", __func__);
> +	isp_composer_deinit(isp_ctx);
> +
> +	return 0;
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> +	struct device *dev = &p1_dev->pdev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
contain some extra bytes at the end.)

> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> +		return;
> +
> +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> +		dev_dbg(dev, "ack frame_num:%d",
> +			ipi_msg->ack_info.frame_seq_no);
> +		atomic_set(&isp_ctx->composed_frame_id,
> +			   ipi_msg->ack_info.frame_seq_no);

I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
frames and we should handle that somehow.

> +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> +	}
> +
> +	atomic_dec_return(&isp_ctx->ipi_occupied);
> +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +}
> +
> +int isp_composer_init(struct device *dev)
> +{
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp_pdev,
> +			       SCP_IPI_ISP_CMD,
> +			       isp_composer_handler,
> +			       isp_ctx);
> +	if (ret)
> +		return ret;
> +
> +	atomic_set(&isp_ctx->cmd_queued, 0);
> +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> +	atomic_set(&isp_ctx->composing_frame, 0);
> +	atomic_set(&isp_ctx->ipi_occupied, 0);
> +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> +
> +	mutex_lock(&isp_ctx->lock);
> +	if (!isp_ctx->composer_tx_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> +		isp_ctx->composer_tx_thread.thread =
> +			kthread_run(isp_composer_tx_work, isp_ctx,
> +				    "isp_composer_tx");
> +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_tx_thread.thread = NULL;
> +			goto nomem;

Why nomem?

> +		}
> +	} else {
> +		dev_warn(dev, "old tx thread is existed\n");

This shouldn't be possible to happen.

> +	}
> +
> +	if (!isp_ctx->composer_deinit_thread.thread) {
> +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> +		isp_ctx->composer_deinit_thread.thread =
> +			kthread_run(isp_composer_deinit_work, isp_ctx,
> +				    "isp_composer_deinit_work");

Why do we need to deinit from another kthread?

> +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> +			dev_err(dev, "unable to start kthread\n");
> +			isp_ctx->composer_deinit_thread.thread = NULL;
> +			goto nomem;
> +		}
> +	} else {
> +		dev_warn(dev, "old rx thread is existed\n");

rx? The code above seems to refer to deinit.

> +	}
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return 0;
> +
> +nomem:
> +	mutex_unlock(&isp_ctx->lock);
> +
> +	return -ENOMEM;

We should return the original error code here.

> +}
> +
> +void isp_composer_enqueue(struct device *dev,
> +			  void *data,
> +			  enum mtk_isp_scp_type type)
> +{
> +	struct mtk_isp_queue_work *isp_composer_work;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);

Just pass p1_dev to this function instead of dev.

> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);

For most of the cases, it should be possible to preallocate this, e.g.
>  - for FRAME, this could be inside the request struct,
>  - for buffer queue it could be inside the buffer struct.

I'd suggest making the caller responsible for allocating if needed.

> +	isp_composer_work->type = type;
> +
> +	switch (type) {
> +	case SCP_ISP_CMD:
> +		memcpy(&isp_composer_work->cmd, data,
> +		       sizeof(isp_composer_work->cmd));
> +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> +			isp_composer_work->cmd.cmd_id);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		atomic_inc(&isp_ctx->cmd_queued);
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> +		break;
> +	case SCP_ISP_FRAME:
> +		memcpy(&isp_composer_work->frameparams, data,
> +		       sizeof(isp_composer_work->frameparams));
> +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> +			isp_composer_work->frameparams.frame_seq_no);
> +
> +		spin_lock(&isp_ctx->composer_txlist.lock);
> +		list_add_tail(&isp_composer_work->list_entry,
> +			      &isp_ctx->composer_txlist.queue);
> +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> +		spin_unlock(&isp_ctx->composer_txlist.lock);
> +
> +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);

The code in both cases is almost exactly the same. The only difference is
the memcpy destination and size and whether isp_ctx->cmd_queued is
incremented or not.

The memcpy will go away if my comment above is addressed and so that would
go down to making the cmd_queued increment conditional.

> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
> +void isp_composer_hw_init(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;

Should we also specify the size of the buffer? Otherwise we could end up
with some undetectable overruns.

> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_meta_config(struct device *dev,
> +			      unsigned int dma)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_config(struct device *dev,
> +			    struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_stream(struct device *dev, int on)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +}
> +
> +void isp_composer_hw_deinit(struct device *dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> +	int ret;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> +
> +	/* Wait for ISP_CMD_DEINIT command is handled done */
> +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> +				 msecs_to_jiffies(2000));
> +	if (ret)
> +		return;
> +
> +	dev_warn(dev, "Timeout & local de-init\n");
> +	isp_composer_deinit(isp_ctx);
> +}
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> new file mode 100644
> index 000000000000..fbd8593e9c2d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> @@ -0,0 +1,207 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2018 MediaTek Inc.
> + */
> +
> +#ifndef _MTK_ISP_SCP_H
> +#define _MTK_ISP_SCP_H
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam-v4l2-util.h"
> +
> +/*
> + * struct img_size - image size information.
> + *
> + * @w: image width, the unit is pixel
> + * @h: image height, the unit is pixel
> + * @xsize: bytes per line based on width.
> + * @stride: bytes per line when changing line.
> + *          Normally, calculate new STRIDE based on
> + *          xsize + HW constrain(page or align).
> + *
> + */
> +struct img_size {
> +	__u32 w;
> +	__u32 h;
> +	__u32 xsize;
> +	__u32 stride;
> +} __packed;
> +
> +/*
> + * struct img_buffer - buffer address information.
> + *
> + * @iova: DMA address for external devices.
> + * @scp_addr: SCP address for external co-process unit.
> + *
> + */
> +struct img_buffer {
> +	__u32 iova;
> +	__u32 scp_addr;
> +} __packed;
> +
> +struct p1_img_crop {
> +	__u32 left;
> +	__u32 top;
> +	__u32 width;
> +	__u32 height;
> +} __packed;
> +
> +struct p1_img_output {
> +	struct img_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	__u8 pixel_byte;
> +	__u32 img_fmt;
> +} __packed;

Please document.

> +
> +/*
> + * struct cfg_in_param - image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: indicate the sensor mode.
> + *              1: continuous
> + *              0: single
> + * @subsample: indicate to enables SOF subsample or not.
> + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: describe input data pattern.
> + * @raw_pixel_id: bayer sequence.
> + * @tg_fps: the fps rate of TG (time generator).
> + * @img_fmt: the image format of input source.
> + * @p1_img_crop: the crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	__u8 continuous;
> +	__u8 subsample;
> +	__u8 pixel_mode;
> +	__u8 data_pattern;
> +	__u8 raw_pixel_id;
> +	__u16 tg_fps;
> +	__u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - the image output parameters of main stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @pure_raw: indicate the image path control.
> + *            1: pure raw
> + *            0: processing raw
> + * @pure_raw_pack: indicate the image is packed or not.
> + *                 1: packed mode
> + *                 0: unpacked mode
> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	/* Bypass main out parameters */
> +	__u8 bypass;
> +	/* Control HW image raw path */
> +	__u8 pure_raw;
> +	/* Control HW image pack function */

No need for these inline comments.

> +	__u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - the image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: indicate this device is enabled or disabled or not .

Remove the space before the period.

> + * @p1_img_output: the output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	/* Bypass resize parameters */

No need for this inline comment.

> +	__u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_meta_out_param - output meta information.
> + *
> + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> + *
> + */
> +struct cfg_meta_out_param {
> +	__u32 enabled_meta_dmas;
> +} __packed;
> +
> +struct p1_config_param {
> +	/* Sensor/TG info */
> +	struct cfg_in_param cfg_in_param;
> +	/* IMGO DMA */
> +	struct cfg_main_out_param cfg_main_param;
> +	/* RRZO DMA */
> +	struct cfg_resize_out_param cfg_resize_param;
> +	/* 3A DMAs and other. */
> +	struct cfg_meta_out_param cfg_meta_param;

Please change the inline comments to a kerneldoc comment at the top.

> +} __packed;
> +
> +struct p1_frame_param {
> +	/* frame sequence number */
> +	__u32 frame_seq_no;
> +	/* SOF index */
> +	__u32 sof_idx;
> +	/* The memory address of tuning buffer from user space */

Ditto.

> +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +struct P1_meta_frame {
> +	__u32 enabled_dma;
> +	__u32 vb_index;
> +	struct img_buffer meta_addr;
> +} __packed;
> +
> +struct isp_init_info {
> +	__u8 hw_module;
> +	struct img_buffer cq_addr;
> +} __packed;
> +
> +struct isp_ack_info {
> +	__u8 cmd_id;
> +	__u32 frame_seq_no;
> +} __packed;
> +
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_CONFIG_META,
> +	ISP_CMD_ENQUEUE_META,
> +	ISP_CMD_RESERVED,
> +};
> +
> +struct mtk_isp_scp_p1_cmd {
> +	__u8 cmd_id;
> +	union {
> +		struct isp_init_info frameparam;
> +		struct p1_config_param config_param;
> +		struct cfg_meta_out_param cfg_meta_out_param;
> +		struct P1_meta_frame meta_frame;
> +		__u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +enum mtk_isp_scp_type {
> +	SCP_ISP_CMD = 0,
> +	SCP_ISP_FRAME,
> +};

Please document all the structs and enum above using kerneldoc.

Best regards,
Tomasz


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-10  9:54       ` Tomasz Figa
  (?)
@ 2019-07-18  4:39         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-18  4:39 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi, Tomasz:

On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and seven video devices during
> > initialization. Moreover, it also connects with sensor and
> > seninf drivers with V4L2 async APIs.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "media: support Mediatek sensor interface driver"[1].
> > 
> > ISP P1 sub-device communicates with seninf sub-device with CIO.
> > 
> > [1]. media: support Mediatek sensor interface driver
> > https://patchwork.kernel.org/cover/10979135/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
> >  3 files changed, 1848 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > 
> 
> Thanks for the patch. Please see my comments inline.
> 
> [snip]
> 

Appreciate your comments on this patch.
Please check my replied inline.

> > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > +				      struct media_request *new_req)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > +
> > +	if (!cam_dev->streaming) {
> > +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > +		spin_lock(&cam_dev->req_lock);
> > +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > +		spin_unlock(&cam_dev->req_lock);
> > +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > +		return;
> > +	}
> > +
> > +	/* Normal enqueue flow */
> > +	if (new_req) {
> > +		mtk_isp_req_enqueue(dev, new_req);
> > +		return;
> > +	}
> > +
> > +	/* Flush all media requests wehen first stream on */
> > +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > +		list_del(&req->list);
> > +		mtk_isp_req_enqueue(dev, &req->req);
> > +	}
> > +}
> 
> This will have to be redone, as per the other suggestions, but generally one
> would have a function that tries to queue as much as possible from a list to
> the hardware and another function that adds a request to the list and calls
> the first function.
> 

We revised this function as below.
First to check the en-queue conditions:
a. stream on 
b. The composer buffers in SCP are 3, so we only could has 3 jobs
at the same time.


Second, try to en-queue the frames in the pending job if possible and
move them into running job list if possible.

The request has been inserted into pending job in mtk_cam_req_validate
which is used to validate media_request.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam_dev->running_job_count);
	unsigned long flags;

	if (!cam_dev->streaming ||
	    buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam_dev->pending_job_lock);
	list_for_each_entry_safe(req, req_prev,
				 &cam_dev->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam_dev->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam_dev->pending_job_lock);

	list_for_each_entry_safe(req, req_prev,
				 &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam_dev->running_job_lock, flags);
		list_add_tail(&req->list, &cam_dev->running_job_list);
		spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);

		mtk_isp_req_enqueue(cam_dev, req);
	}
}


> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> > +
> > +	vb2_request_queue(req);
> > +	mtk_cam_req_try_isp_queue(cam_dev, req);
> 
> Looks like this driver is suffering from versy similar problems in request
> handling as the DIP driver used to.
> 
> I'd prefer to save my time and avoid repeating the same comments, so please
> check my comments for the DIP driver and apply them to this one too:
> 
> https://patchwork.kernel.org/patch/10905223/
> 

Yes, we will follow the same design of DIP and replace this function by
vb2_request_queue and defined new request structure. 

/*
 * struct mtk_cam_dev_request - MTK camera device request.
 *
 * @req: Embedded struct media request.
 * @frame_params: The frame info. & address info. of enabled DMA nodes.
 * @frame_work: work queue entry for frame transmission to SCP.
 * @list: List entry of the object for @struct mtk_cam_dev:
 *        pending_job_list or running_job_list.
 * @buf_count: Buffer count in this request.
 *
 */
struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};


> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
> 
> Doesn't this function return bits not bytes?
> 

Yes, the unit sould be bits, not bytes.
We will rename this function to get_pixel_bits.

static unsigned int get_pixel_bits(unsigned int pix_fmt)

> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> > +					__u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> 
> The __ prefixed types should be used only inside UAPI. Please change the
> driver to use the normal ones.
> 

Ok, we will fix our usage in our driver source code.

> > +
> > +	width = ALIGN(width, 4);
> 
> If there is some alignment requirement for width, it should be handled by
> TRY_/S_FMT and here we should already assume everything properly aligned.
> 

We will follow your suggestion to move this code login in TRY_/S_FMT
functions.

> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +
> > +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> > +				       __u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> 
> Ditto.
> 

Will fix.

> > +	stride = DIV_ROUND_UP(width * 3, 2);
> 
> Could we introduce a local variable for this intermediate value, so that its
> name could explain what the value is?
> 
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +
> > +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > +		stride = ALIGN(stride, 4);
> 
> Is it expected that only the F10 format needs this alignment?
> 

yes, if the pixel bits of image format is 10, the byte alignment of bpl
should be 4. Otherwise, it is 8. We will revise this and add more
comments.

/* 4 bytes alignment for 10 bit other are 8 bytes alignment */
	if (pixel_bits == 10)
		bpl = ALIGN(bpl, 4);
	else
		bpl = ALIGN(bpl, 8);

> > +
> > +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +static void
> > +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> > +			   struct v4l2_pix_format_mplane *dest_fmt,
> > +			   unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> 
> Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?
> 

Has fixed it.
Btw we will only support 1 plane, there is no need for plane_size.

> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = img_cal_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> 
> This memset is not needed. The core clears the reserved fields
> automatically:
> 
> https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559
> 
> (We may want to backport the patch that added that to our 4.19 branch.)
> 

Ok, we will remove both memset functions in this function.


> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> 
> Hmm, we only seem to support 1 plane raw formats in this driver. Does the
> hardware support any formats with more than 1 plane? If not, all the code
> should be simplified to just assume 1 plane.
> 

No, MTK P1 ISP HW only supports raw formats with 1 plane.
We will revise our source codes to support only 1 plane.

> > +}
> > +
> > +static void
> > +mtk_cam_dev_set_img_fmt(struct device *dev,
> > +			struct v4l2_pix_format_mplane *dest_fmt,
> > +			const struct v4l2_pix_format_mplane *src_fmt,
> > +			unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> 
> Given that src_fmt should already be validated and have any fields adjusted
> to match the driver requirements, wouldn't all the lines above be equivalent
> to *dest_fmt = *src_fmt?
> 
> We probably want to move setting all the constant fields to
> mtk_cam_vidioc_try_fmt().
> 

Ok, we will remove mtk_cam_dev_set_img_fmt function
and use *dest_fmt = *src_fmt directly in the caller.
Moreover, all the constant fields are moved to mtk_cam_vidioc_try_fmt()
function.

> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
> 
> This should have been called already before this function was called,
> because src_fmt should be already expected to contain valid settings. In
> fact, this is already called in mtk_cam_vidioc_try_fmt().
> 

Ok, we will revise this.

> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct device *dev,
> 
> Please don't pass struct device pointer around, but instead just the main
> driver data struct, which should be much more convenient for accessing
> various driver data. Please fix the other functions as well.
> 

Ok, we will revise this coding style in our source codes.

> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_set_img_fmt(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> 
> We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
> v4l2_format struct and have any incorrect fields replaced by
> mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
> v4l2_format at runtime.
> 

Ok, we will revise this.

> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +static int mtk_cam_isp_open(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	int ret;
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	ret = v4l2_fh_open(file);
> > +	if (ret)
> > +		goto unlock;
> > +
> > +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
> 
> Please don't power on open. Normally applications keep the device nodes open
> all the time, so they would keep everything powered on.
> 
> Normally this should be done as late as possible, ideally when starting the
> streaming.
> 

Ok, we will remove this function and just call 4l2_fh_open(file)
function.

> > +	if (ret)
> > +		dev_err(dev, "%s fail:%d", __func__, ret);
> > +
> > +unlock:
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_isp_release(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> > +	vb2_fop_release(file);
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> If we remove power handling from open and release, we should be able to just
> use v4l2_fh_open() and vb2_fop_release() directly in the
> v4l2_file_operations struct.
> 

Ok, we will fix this.

> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct v4l2_subdev *sensor;
> 
> This variable would be unitialized if there is no streaming sensor. Was
> there no compiler warning generated for this?
> 

No, there is no compiler warning.
But, we will assign sensor to NULL to avoid unnecessary compiler warning
with different compiler options.

> > +
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "Sensor is not connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ok, we will move this logic check in the mtk_cam_sd_s_stream function.

> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> 
> I don't think -EPERM is a good error code here. It's about a missing seninf
> device, so perhaps -ENODEV?
> 

Fix it in next patch.

> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> > +	if (!cam_dev->sensor)
> > +		return -EPERM;
> 
> > -ENODEV
> 
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> 
> Maybe just return ret?
> 

Fix it in next patch.

> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> > +	isp_composer_stream(dev, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +
> > +	return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ditto.

> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	isp_composer_stream(dev, 0);
> 
> Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
> can't guarantee that the hardware releases all the memory that we're going
> to free once this function returns.
> 

We will add  below functions.
1. Stream off ISP HW
2. Stop ISP HW
3. Clear all pending & running request lists.

	cam_dev->streaming = false;
	mtk_isp_stream(cam_dev, 0);
	mtk_isp_hw_release(cam_dev);
	mtk_cam_dev_req_clear(cam_dev);

	dev_dbg(dev, "streamed off Pass 1\n");

> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> > +
> > +	return on ? mtk_isp_power_init(cam_dev) :
> > +		    mtk_isp_power_release(&cam_dev->pdev->dev);
> 
> s_power is a historical thing and we shouldn't be implementing it. Instead,
> we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
> whenever we start and stop streaming respectively.
> 

Ok, we will remove this callback function.

> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> 
> I assume this check is needed, because the pads with higher indexes are not
> video nodes? If so, a comment would be helpful here.
> 

Yes, we will new comment as below.

	/*
	 * Check video nodes is enabled by link setup.
	 * The pad index of video node should be less than       
         * MTK_CAM_P1_TOTAL_NODES.
	 */
	if (pad < MTK_CAM_P1_TOTAL_NODES)
		cam_dev->vdev_nodes[pad].enabled =
			!!(flags & MEDIA_LNK_FL_ENABLED);

> > +		cam_dev->vdev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> 
> This can be folded into the declaration.
> 

Fix it in next patch.

> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		buf->vbb.request_fd,
> > +		buf->vbb.vb2_buf.index);
> > +
> > +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > +	if (vb->vb2_queue->uses_requests)
> > +		return;
> 
> I'd suggest removing non-request support from this driver. Even if we end up
> with a need to provide compatibility for non-request mode, then it should be
> built on top of the requests mode, so that the driver itself doesn't have to
> deal with two modes.
> 

The purpose of non-request function in this driver is needed by
our camera middle-ware design. It needs 3A statistics buffers before
image buffers en-queue. So we need to en-queue 3A statistics with
non-request mode in this driver. After MW got the 3A statistics data, it
will en-queue the images, tuning buffer and other meta buffers with
request mode. Based on this requirement, do you have any suggestion?
For upstream driver, should we only consider request mode?

> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *smem_dev = cam_dev->smem_dev;
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> 
> Just a reminder that this will have to be reworked according to my comments
> for the memory allocation patch.
> 

Yes, we have revised this implementation according to the review of
below patch set.

https://patchwork.kernel.org/patch/10985833/

> > +
> > +	/* scp address is only valid for meta input buffer */
> > +	if (node->desc.smem_alloc)
> > +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> > +							      buf->daddr);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size)
> > +		return -EINVAL;
> 
> For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
> Otherwise we could get not enough or invalid data.
> 

Fixed in next patch.

> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> 
> This shouldn't be called on OUTPUT buffers.
> 

Ditto.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> 
> We should also verify that *num_planes == 1, as we don't support more
> planes in this driver.
> 

Ok, here is new check logic.

	if (*num_planes) {
		if (sizes[0] < size || *num_planes != 1)
			return -EINVAL;
	} else {
		*num_planes = 1;
		sizes[0] = size;
	}


> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct mtk_cam_dev_request *req, *req0;
> > +	struct media_request_object *obj, *obj0;
> > +	struct vb2_buffer *vb;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> 
> nit: One would normally call the second argument "prev", or "b_prev".
> 

Ok, we will follow this coding convention in our source codes. 

> > +		vb = &b->vbb.vb2_buf;
> > +		if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> We shouldn't need to check the buffer state.
> 

Fixed in next patch.

> > +			vb2_buffer_done(vb, state);
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	spin_lock(&cam_dev->req_lock);
> > +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
> 
> nit: Ditto.
> 

Fixed in next patch.

> > +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
> 
> Need to check if the object is a buffer.
> 

Fixed in next patch.

> > +			vb = container_of(obj, struct vb2_buffer, req_obj);
> > +			if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> vb->state shouldn't be accessed directly from the drivers.
> 

nit: Ditto.

> Generally, the need to check the state here would suggest that there is
> something wrong with how the driver manages the requests. The list that is
> being iterated here shouldn't contain any requests that have buffers that
> aren't active. That will be achieved if my comments for the request handling
> in the DIP driver are applied to this driver as well.
> 
> > +				vb2_buffer_done(vb, state);
> > +		}
> > +		list_del(&req->list);
> > +	}
> > +	spin_unlock(&cam_dev->req_lock);
> > +
> > +	if (node->vbq.uses_requests)
> > +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> 
> How is this synchronized with mtk_cam_media_link_setup()?
> 

We will follow your suggestion and below is our proposal for this
function.

1. Use !cam_dev->pipeline.streaming_count to decide the first node to
stream-on.
2.a If yes, do the following steps
    2.a-1 Call media_pipeline_start function to prevent the link
configuration changes.
    2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
video nodes are enabled and save it into cam_dev->enabled_node_count.
    2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
user has called stream-on API
2.b jump step 3.

3. Use cam_dev->streamed_node_count to track how many video nodes are
streamed by user space.
4. Check all enabled video nodes are streamed or not based on
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
5. If yes, call s_stream on for P1 sub-device

Do you think it is reasonable?

> > +		dev_err(dev, "Node:%d is not enable\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_no_link;
> > +	}
> > +
> > +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> > +		atomic_read(&cam_dev->streamed_node_count), node_count);
> > +
> > +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> > +		return 0;
> 
> How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
> between calls to this function on different video nodes?
> 

Ditto.

> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret) {
> > +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> > +		goto fail_start_pipeline;
> > +	}
> > +
> 
> Related to the above comment: If we start the media pipeline when we start
> streaming on the first node, we would naturally prevent the link
> configuration changes until the last node stops streaming (as long as the
> link is not DYNAMIC). Note that it would only mark the entities as
> streaming, but it wouldn't call their s_stream, which I believe is exactly
> what we would need to solve the problem above.
> 

Ditto.

> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> > +		goto fail_stream_on;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_stream_on:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_start_pipeline:
> > +	atomic_dec(&cam_dev->streamed_node_count);
> > +fail_no_link:
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	if (!node->enabled)
> > +		return;
> 
> It shouldn't be possible for this to happen, because nobody could have
> called start_streaming on a disabled node.
> 

Will remove in next patch.

> > +
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> 
> Shouldn't we stop streaming first, so that the hardware operation is
> cancelled and any buffers owned by the hardware are released?
> 

For this function, below is the new code flow.

1. Check the first node to stream off based on 
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
2. If yes, call all s_stream off for P1 sub-device
3. Call mtk_cam_vb2_return_all_buffers for each node
4. Check the last node to stream off
5. If yes, call media_pipeline_stop to allow user space
to perform link configuration changes, such as disable link.

But, for step 5, is it too late for end user to disable link?
For example, for first node, it has called stream off but
can't call disable link until the last node is stream off?

> > +
> > +	dev_dbg(dev, "%s: count info:%d", __func__,
> > +		cam_dev->subdev.entity.stream_count);
> > +
> > +	/* Check the first node to stream-off */
> > +	if (!cam_dev->subdev.entity.stream_count)
> > +		return;
> > +
> > +	media_pipeline_stop(&node->vdev.entity);
> > +
> > +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +		dev_err(dev, "failed to stop streaming\n");
> > +}
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> 
> This would end up being called multiple times, once for each video node.
> Instead, this should be called explicitly by the driver when it completed
> the request - perhaps in the frame completion handler?
> 
> With that, we probably wouldn't even need this callback.
> 

First, if we don't implement this callback function, we will receive
kernel warning as below.

https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420

Second, this function is only be called in __vb2_queue_cancel function.
Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
So could we just implement dummy empty function?

 * @buf_request_complete: a buffer that was never queued to the driver
but is
 *			associated with a queued request was canceled.
 *			The driver will have to mark associated objects in the
 *			request as completed; required if requests are
 *			supported.


> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> 
> We could just use dev_driver_name(cam_dev->dev) for both.
> 
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> 
> We should just store dev in cam_dev.
> 

Will fix in next patch.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> 
> Is the set of formats available always the same regardless of the sensor
> format?
> 

Yes, ISP P1 HW output formats are always available without impact
by sensor formats. 

> > +	f->flags = 0;
> 
> We need f->description too.
> 

For this description, do you suggest 1). we fill this field in this
function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?

https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152

Basically, we prefer method 1.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (!node->desc.num_fmts)
> > +		return -EINVAL;
> 
> When would that condition happen?
> 

Will remove this in next patch.

> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	const struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> 
> Don't use __ types in implementation, they are here for UAPI purposes. There
> is u32, which you could use instead, but for width and height you don't need
> explicit size, so unsigned int should be good.
> 

Ok, we will revise this in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc, in_fmt);
> 
> We shouldn't just load a default format. This function should validate all
> the fields one by one and adjust them to something appropriate.
> 

For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
a. width
b. height
c. pixelformat
d. plane_fmt
    - sizeimage
    - bytesperline
e. num_planes
Other fields are consider constant.

So if the user space passes one pixel format with un-supported, we will
apply the default format firstly and adjust width, height, sizeimage,
and bytesperline. We will focus on validate width & height.
Is it ok?

> > +	}
> 
> CodingStyle: No braces if both if and else bodies have only 1 statement
> each.
> 

Will fix coding style in next patch.

> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> 
> Shouldn't we clamp this with some maximum value too?
> 

Ok, will revise as below:

	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MAX_HEIGHT, IMG_MIN_HEIGHT);

> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> 
> Ditto.
> 

Ditto.

> > +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> > +				   &in_fmt->fmt.pix_mp, node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> 
> I think this should rather be something like vb2_queue_is_busy(), which
> would prevent format changes if buffers are allocated.
> 

Since vb2_queue_is_busy is static function, would we paste its
implementation in this function to check like this?

	if (node->vdev.queue->owner &&
		node->vdev.queue->owner != file->private_data) {
		dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
		return -EBUSY;
	}

> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +				     struct v4l2_input *input)
> > +{
> > +	if (input->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> > +				  unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_input(struct file *file,
> > +				  void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +	.s_power = mtk_cam_sd_s_power,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> 
> nit: mtk_cam_media_entity_ops?
> 

Rename in next patch.

> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = mtk_cam_isp_open,
> > +	.release = mtk_cam_isp_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> 
> nit: Those are media ops, so perhaps just mtk_cam_media_ops?
> 

Rename in next patch.

> > +	.link_notify = v4l2_pipeline_link_notify,
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> 
> Could we replace any use of this macro with dev_driver_string(dev) and then
> delete the macro? The less name strings in the driver the better, as there
> is less change for confusing the userspace if few different names are used
> at the same time.
> 

Ok, revised in next patch.

> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +
> > +	return media_device_register(media_dev);
> > +}
> > +
> > +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
> 
> Would it make sense to pass node as an argument to this function instead of
> (or in addition to) i?
> 

Ok, revised in next patch.

> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
> 
> Why not call it capture instead and avoid the inversion?
> 

Ok, we will revised as below.

unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);

> > +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> > +	int ret;
> > +
> > +	cam_dev->subdev_pads[i].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->enabled = false;
> 
> Are all the nodes optional? If there is any required node, it should be
> always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.
> 

Ok, MTK_CAM_P1_MAIN_STREAM_OUT is required node, others are optional.
We will enable TK_CAM_P1_MAIN_STREAM_OUT with MEDIA_LNK_FL_IMMUTABLE |
MEDIA_LNK_FL_ENABLED.

> > +	node->id = i;
> > +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> 
> Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?
> 

Yes, it is wrong and fix with below statement.

node->vdev_pad.flags = output ? MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;


> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     &node->desc,
> > +				     &node->vdev_fmt);
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->vdev_fmt.type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam_dev->smem_dev;
> > +	} else {
> > +		vbq->dev = &cam_dev->pdev->dev;
> > +	}
> > +
> > +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		vdev->entity.function =
> > +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> 
> This is a video node, so it's just a DMA, not a processing entity. I believe
> all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.
> 

Ok, it is fixed.

> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +	/* Put the process hub sub device in the vb2 private data */
> 
> What is "process hub" and what "sub device" is this about?
> 

We will drop this comment.

> > +	vbq->drv_priv = cam_dev;
> > +	vbq->lock = &node->lock;
> > +	vbq->supports_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_vb2_queue;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->lock;
> > +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.ops = NULL;
> > +	/* Enable private control for image video devices */
> > +	if (node->desc.image) {
> > +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +		vdev->ctrl_handler = &node->ctrl_handler;
> > +	}
> > +	video_set_drvdata(vdev, cam_dev);
> > +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vdev;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam_dev->subdev.entity,
> > +					    i, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +					    i, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> > +	if (ret)
> > +		goto fail_link;
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> > +	INIT_LIST_HEAD(&node->pending_list);
> 
> This should be all initialized before registering the video device.
> Otherwise userspace could open the device before these are initialized.
> 

Fixed in next patch.

> > +
> > +	return 0;
> > +
> > +fail_link:
> > +	video_unregister_device(vdev);
> > +fail_vdev:
> > +	vb2_queue_release(vbq);
> > +fail_vb2_queue:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> 
> This function doesn't have anything to do with mem2mem. How about
> mtk_cam_v4l2_register()?
> 
> Perhaps it would make sense to move any media related code into into
> mtk_cam_media_register(), keep only V4L2 related code here and have the
> caller call the former first and then this one, rather than having such deep
> sequence of nested calls, which makes the driver harder to read.
> 

Fixed in next patch.

> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> 
> How about just storing dev, instead of pdev in the struct? Also, calling the
> argument "cam", would make it as short as cam->dev.
> 

Ok, we will revise this in next patch.

> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> 
> An info message should be useful to the user in some way. Printing kernel
> pointers isn't useful. Something like "registered media0" could be useful to
> let the user know which media device is associated with this driver if there
> is more than one in the system.
> 

Here is the new log info.

dev_info(dev, "media%d register",cam->media_dev.devnode->minor);


> > +
> > +	/* Set up v4l2 device */
> > +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> 
> Please call the labels after the cleanup step that needs to be done. It
> makes it easier to spot any ordering errors.
> 

Will fix in next patch.

> > +	}
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> 
> Same as above.
> 

Ditto.

dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);

> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> 
> Stray ":" at the end of the message.
> 

Fixed in next patch.

> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		ret = mtk_cam_video_register_device(cam_dev, i);
> > +		if (ret)
> > +			goto fail_video_register;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_video_register:
> > +	i--;
> 
> This could be moved into the for clause, as the initialization statement.
> 

Fixed in next patch.

> > +	for (; i >= 0; i--) {
> 
> i is unsigned. Did this compile without warnings?
> 
> > +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> > +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> > +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
> 
> Should we move this into mtk_cam_video_unregister_device() to be consistent
> with registration?
> 

Fixed in next patch.

> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> 
> Please print errors only where they actually happen, not at the cleanup.
> 

Fixed in next patch.

> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> 
> nit: Move the declaration inside the for loop, since the variable is only
> used there.
> 

Fixed in next patch.

> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		dev = &cam_dev->vdev_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> 
> This was allocated using devm_kcalloc(), so no need to free it explicitly.
> 

Fixed in next patch.

> > +
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_CIO_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_CIO_PAD_SINK,
> > +				    0);
> > +	if (ret) {
> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> 
> Why not just put the contents of this function inside 
> mtk_cam_dev_notifier_complete()?
> 

Ok, we will mtk_cam_dev_complete() function and move its content into
mtk_cam_dev_notifier_complete().

> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> 
> Should we somehow check that the entity we got is seninf indeed and there
> was no mistake in DT?
> 

How about to check the entity function of seninf device?

if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
	dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
		return -ENODEV;
}

If we need to check DT, may we need to implement this in parse_endpoint
callback function of v4l2_async_notifier_parse_fwnode_endpoints?

> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> 
> bound
> 
> Also please make this dev_dbg().
> 

Fixed in next patch.

> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = NULL;
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> 
> unbound
> 

Fixed in next patch.

> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> > +		NULL);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> 
> Could we print some error messages for the 2 cases above?
> 

Fixed in next patch.

> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> 
> I don't think we need vidioc_*_input. At least the current implementation in
> this patch doesn't seem to do anything useful.
> 
> > +	/* buffer queue management */
> 
> Drop this comment, as it's obvious that the callbacks with "buf" in the name
> are related to buffers, there are some non-buffer callbacks below too (e.g.
> vidioc_subscribe_event) and also the other ops structs below don't have such
> comment.
> 

Fixed in next patch.

> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> 
> PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
> making it 512 * SZ_1K?
> 
> That said, it should normally be just sizeof(struct some_struct_used_here).
> 
> Same for the other entries below.
> 

ok, we will change the size unit from PAGE_SIZE to SZ_1K.
If we finalize the meta structures design, we may change to use sizeof.

> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = IMG_MAX_WIDTH,
> > +			.min_width = IMG_MIN_WIDTH,
> > +			.max_height = IMG_MAX_HEIGHT,
> > +			.min_height = IMG_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = RRZ_MAX_WIDTH,
> > +			.min_width = RRZ_MIN_WIDTH,
> > +			.max_height = RRZ_MAX_HEIGHT,
> > +			.min_height = RRZ_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 1,
> 
> Why not just make it always 0 and move the default format to the beginning
> of stream_out_fmts[]?
> 

Fixed in next patch.

> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[0],
> 
> nit: If you only need to point to these data once, you could define them in
> place, as a compound literal, like
> 

Fixed in next patch.

> >                 .frmsizes = &(struct v4l2_framesizeenum) {
> >                         // initialize here
> >                 },
> 

Ditto

> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[1],
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> 
> link_flags seems to be 0 for all nodes.
> 

Ditto.

> > +		.capture = true,
> 
> We already know this from .buf_type. We can check using the
> V4L2_TYPE_IS_OUTPUT() macro.
> 

Ditto.

> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> 
> I don't think this is correct. The description suggests one specific format
> (local motion vector histogram), so the queue should probably be hardwired
> to that format?
> 

Yes, we will set num_fmts = 1 for meta nodes.

> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> 
> Where does this number come from?
> 

The default maximum VB2 buffer count is 32.
In order to limit memory usage, we like to limit the maximum buffer
counts in the driver layer. The maximum buffer count is estimated
according to our camera MW.

#define VB2_MAX_FRAME	(32)

> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
> 
> < ARRAY_SIZE(output_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
> 
> < ARRAY_SIZE(capture_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> 
> Do we need this additional mtk_cam_dev struct? Couldn't we just use
> mtk_isp_p1 here?
> 

We remove pdev field and add dev field in mtk_cam_dev struct.

> > +	mtk_cam_dev_queue_setup(cam_dev);
> > +	/* v4l2 sub-device registration */
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> 
> This debugging message doesn't seem very useful. Please remove.
> 

Fixed in next patch.

> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret) {
> > +		mtk_cam_v4l2_unregister(cam_dev);
> 
> Please use an error path on the bottom of the function instead. With goto
> labels named after the next clean-up step that needs to be performed.
> 

Fixed in next patch.

> > +		return ret;
> > +	}
> > +
> > +	spin_lock_init(&cam_dev->req_lock);
> > +	INIT_LIST_HEAD(&cam_dev->req_list);
> > +	mutex_init(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> 
> "release" is normally used for file_operations. How about "cleanup"?
> 

Fixed in next patch.

> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	mutex_destroy(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> I'd suggest moving any generic API implementation code (platform_device,
> V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
> level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
> mtk_cam_hw.c.
> 

Fixed in next patch.

> > +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..825cdf20643a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,173 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
> 
> Maybe it's not a critical thing, but generally it's a good practice to just
> explicitly specific this name somewhere, e.g. struct
> platform_driver::driver::name and then just refer to dev_driver_name(). It
> makes it easier to make sure that the name stays the same everywhere.
> 

Fixed in next patch.

> > +
> > +#define MTK_CAM_P1_META_IN_0			0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> 
> Since these are just some utility definitions, we can use enum instead of
> assigning the values by hand.
> 

We will revise these definitions as below.

/* ID enum value for struct mtk_cam_dev_node_desc:id */
enum  {
	MTK_CAM_P1_META_IN_0 = 0,
	MTK_CAM_P1_MAIN_STREAM_OUT,
	MTK_CAM_P1_PACKED_BIN_OUT,
	MTK_CAM_P1_META_OUT_0,
	MTK_CAM_P1_META_OUT_1,
	MTK_CAM_P1_META_OUT_2,
	MTK_CAM_P1_META_OUT_3,
	MTK_CAM_P1_TOTAL_NODES
};

> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> > +#define MTK_CAM_P1_META_OUT_0			3
> > +#define MTK_CAM_P1_META_OUT_1			4
> > +#define MTK_CAM_P1_META_OUT_2			5
> > +#define MTK_CAM_P1_META_OUT_3			6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE		6
> 
> Ditto.
> 

Ditto.

> > +
> > +#define MTK_CAM_P1_TOTAL_NODES			7
> 
> Please just add the two totals together rather than manually specifying the
> value.
> 

Ditto.

> > +
> > +struct mtk_cam_dev_request {
> > +	struct media_request	req;
> > +	struct list_head	list;
> > +};
> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +	unsigned int		node_id;
> > +};
> 
> Could you add kerneldoc comments for the 2 structs?
> 

Fixed in next patch.

> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + * @frmsizes:	supported frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> 
> vdev_pad?
> 
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> 
> Please fix the order of the documentation to match the order of the struct.
> 

Fixed in next patch.

> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct video_device vdev;
> 
> Not documented above.
> 

Fixed in next patch.

> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.
Btw, if we remove this, we will got complain from checkpatch.pl script.

> > +	struct mutex lock;
> > +	/* protect for pending_list */
> 
> It's already documented in the kerneldoc comment.
> 

Ditto.

> > +	spinlock_t slock;
> 
> How about calling it pending_list_lock?
> 

We will rename to buf_list to track all en-queue buffers in this video
node.

struct mtk_cam_video_device {
	unsigned int id;
	unsigned int enabled;
	struct v4l2_format vdev_fmt;
	struct mtk_cam_dev_node_desc desc;
	struct video_device vdev;
	struct media_pad vdev_pad;
	struct vb2_queue vbq;
	struct v4l2_ctrl_handler ctrl_handler;
	struct list_head buf_list;
	struct mutex lock;
	spinlock_t buf_list_lock;
};

> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @lock:       The mutex protecting video device open/release operations
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> > + * @req_list:	Lins to keep media requests before streaming on
> > + * @req_lock:	Protect the req_list data
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> 
> This probably isn't the best place for graph topology description. I think
> we actually want a separate documentation file for this, similar to
> Documentation/media/v4l-drivers/ipu3.rst.
> 

Ok, we will drop our graph topology comment & discuss how to come out
another separate document.

> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	/* protect video device open/release operations */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	struct mutex lock;
> > +	unsigned int streaming:1;
> > +	atomic_t streamed_node_count;
> > +	struct list_head req_list;
> > +	/* protect for req_list */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	spinlock_t req_lock;
> 
> How about calling it req_list_lock?
> 

Below is new mtk_cam_dev structure.
We will use job to handle request.

struct mtk_cam_dev {
	struct device *dev;
	struct device *smem_dev;
	struct media_pipeline pipeline;
	struct media_device media_dev;
	struct v4l2_subdev subdev;
	struct v4l2_device v4l2_dev;
	struct v4l2_async_notifier notifier;
	struct media_pad *subdev_pads;
	struct v4l2_ctrl_handler ctrl_handler;
	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
	struct v4l2_subdev *seninf;
	struct v4l2_subdev *sensor;
	struct mutex lock;
	unsigned int streaming:1;
	unsigned int enabled_dmas;
	unsigned int enabled_node_count;
	atomic_t streamed_node_count;
	struct list_head pending_job_list;
	spinlock_t pending_job_lock;
	struct list_head running_job_list;
	spinlock_t running_job_lock;
	atomic_t running_job_count;
};


> Best regards,
> Tomasz
> 

Thanks again for your many inputs on this patch.
It is helpful for us.

Best regards,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-18  4:39         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-18  4:39 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi, Tomasz:

On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and seven video devices during
> > initialization. Moreover, it also connects with sensor and
> > seninf drivers with V4L2 async APIs.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "media: support Mediatek sensor interface driver"[1].
> > 
> > ISP P1 sub-device communicates with seninf sub-device with CIO.
> > 
> > [1]. media: support Mediatek sensor interface driver
> > https://patchwork.kernel.org/cover/10979135/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
> >  3 files changed, 1848 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > 
> 
> Thanks for the patch. Please see my comments inline.
> 
> [snip]
> 

Appreciate your comments on this patch.
Please check my replied inline.

> > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > +				      struct media_request *new_req)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > +
> > +	if (!cam_dev->streaming) {
> > +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > +		spin_lock(&cam_dev->req_lock);
> > +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > +		spin_unlock(&cam_dev->req_lock);
> > +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > +		return;
> > +	}
> > +
> > +	/* Normal enqueue flow */
> > +	if (new_req) {
> > +		mtk_isp_req_enqueue(dev, new_req);
> > +		return;
> > +	}
> > +
> > +	/* Flush all media requests wehen first stream on */
> > +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > +		list_del(&req->list);
> > +		mtk_isp_req_enqueue(dev, &req->req);
> > +	}
> > +}
> 
> This will have to be redone, as per the other suggestions, but generally one
> would have a function that tries to queue as much as possible from a list to
> the hardware and another function that adds a request to the list and calls
> the first function.
> 

We revised this function as below.
First to check the en-queue conditions:
a. stream on 
b. The composer buffers in SCP are 3, so we only could has 3 jobs
at the same time.


Second, try to en-queue the frames in the pending job if possible and
move them into running job list if possible.

The request has been inserted into pending job in mtk_cam_req_validate
which is used to validate media_request.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam_dev->running_job_count);
	unsigned long flags;

	if (!cam_dev->streaming ||
	    buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam_dev->pending_job_lock);
	list_for_each_entry_safe(req, req_prev,
				 &cam_dev->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam_dev->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam_dev->pending_job_lock);

	list_for_each_entry_safe(req, req_prev,
				 &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam_dev->running_job_lock, flags);
		list_add_tail(&req->list, &cam_dev->running_job_list);
		spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);

		mtk_isp_req_enqueue(cam_dev, req);
	}
}


> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> > +
> > +	vb2_request_queue(req);
> > +	mtk_cam_req_try_isp_queue(cam_dev, req);
> 
> Looks like this driver is suffering from versy similar problems in request
> handling as the DIP driver used to.
> 
> I'd prefer to save my time and avoid repeating the same comments, so please
> check my comments for the DIP driver and apply them to this one too:
> 
> https://patchwork.kernel.org/patch/10905223/
> 

Yes, we will follow the same design of DIP and replace this function by
vb2_request_queue and defined new request structure. 

/*
 * struct mtk_cam_dev_request - MTK camera device request.
 *
 * @req: Embedded struct media request.
 * @frame_params: The frame info. & address info. of enabled DMA nodes.
 * @frame_work: work queue entry for frame transmission to SCP.
 * @list: List entry of the object for @struct mtk_cam_dev:
 *        pending_job_list or running_job_list.
 * @buf_count: Buffer count in this request.
 *
 */
struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};


> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
> 
> Doesn't this function return bits not bytes?
> 

Yes, the unit sould be bits, not bytes.
We will rename this function to get_pixel_bits.

static unsigned int get_pixel_bits(unsigned int pix_fmt)

> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> > +					__u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> 
> The __ prefixed types should be used only inside UAPI. Please change the
> driver to use the normal ones.
> 

Ok, we will fix our usage in our driver source code.

> > +
> > +	width = ALIGN(width, 4);
> 
> If there is some alignment requirement for width, it should be handled by
> TRY_/S_FMT and here we should already assume everything properly aligned.
> 

We will follow your suggestion to move this code login in TRY_/S_FMT
functions.

> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +
> > +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> > +				       __u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> 
> Ditto.
> 

Will fix.

> > +	stride = DIV_ROUND_UP(width * 3, 2);
> 
> Could we introduce a local variable for this intermediate value, so that its
> name could explain what the value is?
> 
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +
> > +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > +		stride = ALIGN(stride, 4);
> 
> Is it expected that only the F10 format needs this alignment?
> 

yes, if the pixel bits of image format is 10, the byte alignment of bpl
should be 4. Otherwise, it is 8. We will revise this and add more
comments.

/* 4 bytes alignment for 10 bit other are 8 bytes alignment */
	if (pixel_bits == 10)
		bpl = ALIGN(bpl, 4);
	else
		bpl = ALIGN(bpl, 8);

> > +
> > +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +static void
> > +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> > +			   struct v4l2_pix_format_mplane *dest_fmt,
> > +			   unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> 
> Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?
> 

Has fixed it.
Btw we will only support 1 plane, there is no need for plane_size.

> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = img_cal_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> 
> This memset is not needed. The core clears the reserved fields
> automatically:
> 
> https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559
> 
> (We may want to backport the patch that added that to our 4.19 branch.)
> 

Ok, we will remove both memset functions in this function.


> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> 
> Hmm, we only seem to support 1 plane raw formats in this driver. Does the
> hardware support any formats with more than 1 plane? If not, all the code
> should be simplified to just assume 1 plane.
> 

No, MTK P1 ISP HW only supports raw formats with 1 plane.
We will revise our source codes to support only 1 plane.

> > +}
> > +
> > +static void
> > +mtk_cam_dev_set_img_fmt(struct device *dev,
> > +			struct v4l2_pix_format_mplane *dest_fmt,
> > +			const struct v4l2_pix_format_mplane *src_fmt,
> > +			unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> 
> Given that src_fmt should already be validated and have any fields adjusted
> to match the driver requirements, wouldn't all the lines above be equivalent
> to *dest_fmt = *src_fmt?
> 
> We probably want to move setting all the constant fields to
> mtk_cam_vidioc_try_fmt().
> 

Ok, we will remove mtk_cam_dev_set_img_fmt function
and use *dest_fmt = *src_fmt directly in the caller.
Moreover, all the constant fields are moved to mtk_cam_vidioc_try_fmt()
function.

> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
> 
> This should have been called already before this function was called,
> because src_fmt should be already expected to contain valid settings. In
> fact, this is already called in mtk_cam_vidioc_try_fmt().
> 

Ok, we will revise this.

> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct device *dev,
> 
> Please don't pass struct device pointer around, but instead just the main
> driver data struct, which should be much more convenient for accessing
> various driver data. Please fix the other functions as well.
> 

Ok, we will revise this coding style in our source codes.

> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_set_img_fmt(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> 
> We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
> v4l2_format struct and have any incorrect fields replaced by
> mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
> v4l2_format at runtime.
> 

Ok, we will revise this.

> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +static int mtk_cam_isp_open(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	int ret;
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	ret = v4l2_fh_open(file);
> > +	if (ret)
> > +		goto unlock;
> > +
> > +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
> 
> Please don't power on open. Normally applications keep the device nodes open
> all the time, so they would keep everything powered on.
> 
> Normally this should be done as late as possible, ideally when starting the
> streaming.
> 

Ok, we will remove this function and just call 4l2_fh_open(file)
function.

> > +	if (ret)
> > +		dev_err(dev, "%s fail:%d", __func__, ret);
> > +
> > +unlock:
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_isp_release(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> > +	vb2_fop_release(file);
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> If we remove power handling from open and release, we should be able to just
> use v4l2_fh_open() and vb2_fop_release() directly in the
> v4l2_file_operations struct.
> 

Ok, we will fix this.

> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct v4l2_subdev *sensor;
> 
> This variable would be unitialized if there is no streaming sensor. Was
> there no compiler warning generated for this?
> 

No, there is no compiler warning.
But, we will assign sensor to NULL to avoid unnecessary compiler warning
with different compiler options.

> > +
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "Sensor is not connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ok, we will move this logic check in the mtk_cam_sd_s_stream function.

> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> 
> I don't think -EPERM is a good error code here. It's about a missing seninf
> device, so perhaps -ENODEV?
> 

Fix it in next patch.

> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> > +	if (!cam_dev->sensor)
> > +		return -EPERM;
> 
> > -ENODEV
> 
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> 
> Maybe just return ret?
> 

Fix it in next patch.

> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> > +	isp_composer_stream(dev, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +
> > +	return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ditto.

> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	isp_composer_stream(dev, 0);
> 
> Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
> can't guarantee that the hardware releases all the memory that we're going
> to free once this function returns.
> 

We will add  below functions.
1. Stream off ISP HW
2. Stop ISP HW
3. Clear all pending & running request lists.

	cam_dev->streaming = false;
	mtk_isp_stream(cam_dev, 0);
	mtk_isp_hw_release(cam_dev);
	mtk_cam_dev_req_clear(cam_dev);

	dev_dbg(dev, "streamed off Pass 1\n");

> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> > +
> > +	return on ? mtk_isp_power_init(cam_dev) :
> > +		    mtk_isp_power_release(&cam_dev->pdev->dev);
> 
> s_power is a historical thing and we shouldn't be implementing it. Instead,
> we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
> whenever we start and stop streaming respectively.
> 

Ok, we will remove this callback function.

> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> 
> I assume this check is needed, because the pads with higher indexes are not
> video nodes? If so, a comment would be helpful here.
> 

Yes, we will new comment as below.

	/*
	 * Check video nodes is enabled by link setup.
	 * The pad index of video node should be less than       
         * MTK_CAM_P1_TOTAL_NODES.
	 */
	if (pad < MTK_CAM_P1_TOTAL_NODES)
		cam_dev->vdev_nodes[pad].enabled =
			!!(flags & MEDIA_LNK_FL_ENABLED);

> > +		cam_dev->vdev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> 
> This can be folded into the declaration.
> 

Fix it in next patch.

> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		buf->vbb.request_fd,
> > +		buf->vbb.vb2_buf.index);
> > +
> > +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > +	if (vb->vb2_queue->uses_requests)
> > +		return;
> 
> I'd suggest removing non-request support from this driver. Even if we end up
> with a need to provide compatibility for non-request mode, then it should be
> built on top of the requests mode, so that the driver itself doesn't have to
> deal with two modes.
> 

The purpose of non-request function in this driver is needed by
our camera middle-ware design. It needs 3A statistics buffers before
image buffers en-queue. So we need to en-queue 3A statistics with
non-request mode in this driver. After MW got the 3A statistics data, it
will en-queue the images, tuning buffer and other meta buffers with
request mode. Based on this requirement, do you have any suggestion?
For upstream driver, should we only consider request mode?

> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *smem_dev = cam_dev->smem_dev;
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> 
> Just a reminder that this will have to be reworked according to my comments
> for the memory allocation patch.
> 

Yes, we have revised this implementation according to the review of
below patch set.

https://patchwork.kernel.org/patch/10985833/

> > +
> > +	/* scp address is only valid for meta input buffer */
> > +	if (node->desc.smem_alloc)
> > +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> > +							      buf->daddr);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size)
> > +		return -EINVAL;
> 
> For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
> Otherwise we could get not enough or invalid data.
> 

Fixed in next patch.

> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> 
> This shouldn't be called on OUTPUT buffers.
> 

Ditto.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> 
> We should also verify that *num_planes == 1, as we don't support more
> planes in this driver.
> 

Ok, here is new check logic.

	if (*num_planes) {
		if (sizes[0] < size || *num_planes != 1)
			return -EINVAL;
	} else {
		*num_planes = 1;
		sizes[0] = size;
	}


> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct mtk_cam_dev_request *req, *req0;
> > +	struct media_request_object *obj, *obj0;
> > +	struct vb2_buffer *vb;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> 
> nit: One would normally call the second argument "prev", or "b_prev".
> 

Ok, we will follow this coding convention in our source codes. 

> > +		vb = &b->vbb.vb2_buf;
> > +		if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> We shouldn't need to check the buffer state.
> 

Fixed in next patch.

> > +			vb2_buffer_done(vb, state);
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	spin_lock(&cam_dev->req_lock);
> > +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
> 
> nit: Ditto.
> 

Fixed in next patch.

> > +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
> 
> Need to check if the object is a buffer.
> 

Fixed in next patch.

> > +			vb = container_of(obj, struct vb2_buffer, req_obj);
> > +			if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> vb->state shouldn't be accessed directly from the drivers.
> 

nit: Ditto.

> Generally, the need to check the state here would suggest that there is
> something wrong with how the driver manages the requests. The list that is
> being iterated here shouldn't contain any requests that have buffers that
> aren't active. That will be achieved if my comments for the request handling
> in the DIP driver are applied to this driver as well.
> 
> > +				vb2_buffer_done(vb, state);
> > +		}
> > +		list_del(&req->list);
> > +	}
> > +	spin_unlock(&cam_dev->req_lock);
> > +
> > +	if (node->vbq.uses_requests)
> > +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> 
> How is this synchronized with mtk_cam_media_link_setup()?
> 

We will follow your suggestion and below is our proposal for this
function.

1. Use !cam_dev->pipeline.streaming_count to decide the first node to
stream-on.
2.a If yes, do the following steps
    2.a-1 Call media_pipeline_start function to prevent the link
configuration changes.
    2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
video nodes are enabled and save it into cam_dev->enabled_node_count.
    2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
user has called stream-on API
2.b jump step 3.

3. Use cam_dev->streamed_node_count to track how many video nodes are
streamed by user space.
4. Check all enabled video nodes are streamed or not based on
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
5. If yes, call s_stream on for P1 sub-device

Do you think it is reasonable?

> > +		dev_err(dev, "Node:%d is not enable\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_no_link;
> > +	}
> > +
> > +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> > +		atomic_read(&cam_dev->streamed_node_count), node_count);
> > +
> > +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> > +		return 0;
> 
> How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
> between calls to this function on different video nodes?
> 

Ditto.

> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret) {
> > +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> > +		goto fail_start_pipeline;
> > +	}
> > +
> 
> Related to the above comment: If we start the media pipeline when we start
> streaming on the first node, we would naturally prevent the link
> configuration changes until the last node stops streaming (as long as the
> link is not DYNAMIC). Note that it would only mark the entities as
> streaming, but it wouldn't call their s_stream, which I believe is exactly
> what we would need to solve the problem above.
> 

Ditto.

> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> > +		goto fail_stream_on;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_stream_on:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_start_pipeline:
> > +	atomic_dec(&cam_dev->streamed_node_count);
> > +fail_no_link:
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	if (!node->enabled)
> > +		return;
> 
> It shouldn't be possible for this to happen, because nobody could have
> called start_streaming on a disabled node.
> 

Will remove in next patch.

> > +
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> 
> Shouldn't we stop streaming first, so that the hardware operation is
> cancelled and any buffers owned by the hardware are released?
> 

For this function, below is the new code flow.

1. Check the first node to stream off based on 
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
2. If yes, call all s_stream off for P1 sub-device
3. Call mtk_cam_vb2_return_all_buffers for each node
4. Check the last node to stream off
5. If yes, call media_pipeline_stop to allow user space
to perform link configuration changes, such as disable link.

But, for step 5, is it too late for end user to disable link?
For example, for first node, it has called stream off but
can't call disable link until the last node is stream off?

> > +
> > +	dev_dbg(dev, "%s: count info:%d", __func__,
> > +		cam_dev->subdev.entity.stream_count);
> > +
> > +	/* Check the first node to stream-off */
> > +	if (!cam_dev->subdev.entity.stream_count)
> > +		return;
> > +
> > +	media_pipeline_stop(&node->vdev.entity);
> > +
> > +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +		dev_err(dev, "failed to stop streaming\n");
> > +}
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> 
> This would end up being called multiple times, once for each video node.
> Instead, this should be called explicitly by the driver when it completed
> the request - perhaps in the frame completion handler?
> 
> With that, we probably wouldn't even need this callback.
> 

First, if we don't implement this callback function, we will receive
kernel warning as below.

https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420

Second, this function is only be called in __vb2_queue_cancel function.
Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
So could we just implement dummy empty function?

 * @buf_request_complete: a buffer that was never queued to the driver
but is
 *			associated with a queued request was canceled.
 *			The driver will have to mark associated objects in the
 *			request as completed; required if requests are
 *			supported.


> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> 
> We could just use dev_driver_name(cam_dev->dev) for both.
> 
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> 
> We should just store dev in cam_dev.
> 

Will fix in next patch.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> 
> Is the set of formats available always the same regardless of the sensor
> format?
> 

Yes, ISP P1 HW output formats are always available without impact
by sensor formats. 

> > +	f->flags = 0;
> 
> We need f->description too.
> 

For this description, do you suggest 1). we fill this field in this
function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?

https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152

Basically, we prefer method 1.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (!node->desc.num_fmts)
> > +		return -EINVAL;
> 
> When would that condition happen?
> 

Will remove this in next patch.

> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	const struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> 
> Don't use __ types in implementation, they are here for UAPI purposes. There
> is u32, which you could use instead, but for width and height you don't need
> explicit size, so unsigned int should be good.
> 

Ok, we will revise this in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc, in_fmt);
> 
> We shouldn't just load a default format. This function should validate all
> the fields one by one and adjust them to something appropriate.
> 

For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
a. width
b. height
c. pixelformat
d. plane_fmt
    - sizeimage
    - bytesperline
e. num_planes
Other fields are consider constant.

So if the user space passes one pixel format with un-supported, we will
apply the default format firstly and adjust width, height, sizeimage,
and bytesperline. We will focus on validate width & height.
Is it ok?

> > +	}
> 
> CodingStyle: No braces if both if and else bodies have only 1 statement
> each.
> 

Will fix coding style in next patch.

> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> 
> Shouldn't we clamp this with some maximum value too?
> 

Ok, will revise as below:

	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MAX_HEIGHT, IMG_MIN_HEIGHT);

> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> 
> Ditto.
> 

Ditto.

> > +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> > +				   &in_fmt->fmt.pix_mp, node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> 
> I think this should rather be something like vb2_queue_is_busy(), which
> would prevent format changes if buffers are allocated.
> 

Since vb2_queue_is_busy is static function, would we paste its
implementation in this function to check like this?

	if (node->vdev.queue->owner &&
		node->vdev.queue->owner != file->private_data) {
		dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
		return -EBUSY;
	}

> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +				     struct v4l2_input *input)
> > +{
> > +	if (input->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> > +				  unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_input(struct file *file,
> > +				  void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +	.s_power = mtk_cam_sd_s_power,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> 
> nit: mtk_cam_media_entity_ops?
> 

Rename in next patch.

> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = mtk_cam_isp_open,
> > +	.release = mtk_cam_isp_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> 
> nit: Those are media ops, so perhaps just mtk_cam_media_ops?
> 

Rename in next patch.

> > +	.link_notify = v4l2_pipeline_link_notify,
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> 
> Could we replace any use of this macro with dev_driver_string(dev) and then
> delete the macro? The less name strings in the driver the better, as there
> is less change for confusing the userspace if few different names are used
> at the same time.
> 

Ok, revised in next patch.

> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +
> > +	return media_device_register(media_dev);
> > +}
> > +
> > +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
> 
> Would it make sense to pass node as an argument to this function instead of
> (or in addition to) i?
> 

Ok, revised in next patch.

> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
> 
> Why not call it capture instead and avoid the inversion?
> 

Ok, we will revised as below.

unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);

> > +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> > +	int ret;
> > +
> > +	cam_dev->subdev_pads[i].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->enabled = false;
> 
> Are all the nodes optional? If there is any required node, it should be
> always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.
> 

Ok, MTK_CAM_P1_MAIN_STREAM_OUT is required node, others are optional.
We will enable TK_CAM_P1_MAIN_STREAM_OUT with MEDIA_LNK_FL_IMMUTABLE |
MEDIA_LNK_FL_ENABLED.

> > +	node->id = i;
> > +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> 
> Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?
> 

Yes, it is wrong and fix with below statement.

node->vdev_pad.flags = output ? MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;


> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     &node->desc,
> > +				     &node->vdev_fmt);
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->vdev_fmt.type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam_dev->smem_dev;
> > +	} else {
> > +		vbq->dev = &cam_dev->pdev->dev;
> > +	}
> > +
> > +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		vdev->entity.function =
> > +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> 
> This is a video node, so it's just a DMA, not a processing entity. I believe
> all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.
> 

Ok, it is fixed.

> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +	/* Put the process hub sub device in the vb2 private data */
> 
> What is "process hub" and what "sub device" is this about?
> 

We will drop this comment.

> > +	vbq->drv_priv = cam_dev;
> > +	vbq->lock = &node->lock;
> > +	vbq->supports_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_vb2_queue;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->lock;
> > +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.ops = NULL;
> > +	/* Enable private control for image video devices */
> > +	if (node->desc.image) {
> > +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +		vdev->ctrl_handler = &node->ctrl_handler;
> > +	}
> > +	video_set_drvdata(vdev, cam_dev);
> > +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vdev;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam_dev->subdev.entity,
> > +					    i, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +					    i, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> > +	if (ret)
> > +		goto fail_link;
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> > +	INIT_LIST_HEAD(&node->pending_list);
> 
> This should be all initialized before registering the video device.
> Otherwise userspace could open the device before these are initialized.
> 

Fixed in next patch.

> > +
> > +	return 0;
> > +
> > +fail_link:
> > +	video_unregister_device(vdev);
> > +fail_vdev:
> > +	vb2_queue_release(vbq);
> > +fail_vb2_queue:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> 
> This function doesn't have anything to do with mem2mem. How about
> mtk_cam_v4l2_register()?
> 
> Perhaps it would make sense to move any media related code into into
> mtk_cam_media_register(), keep only V4L2 related code here and have the
> caller call the former first and then this one, rather than having such deep
> sequence of nested calls, which makes the driver harder to read.
> 

Fixed in next patch.

> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> 
> How about just storing dev, instead of pdev in the struct? Also, calling the
> argument "cam", would make it as short as cam->dev.
> 

Ok, we will revise this in next patch.

> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> 
> An info message should be useful to the user in some way. Printing kernel
> pointers isn't useful. Something like "registered media0" could be useful to
> let the user know which media device is associated with this driver if there
> is more than one in the system.
> 

Here is the new log info.

dev_info(dev, "media%d register",cam->media_dev.devnode->minor);


> > +
> > +	/* Set up v4l2 device */
> > +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> 
> Please call the labels after the cleanup step that needs to be done. It
> makes it easier to spot any ordering errors.
> 

Will fix in next patch.

> > +	}
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> 
> Same as above.
> 

Ditto.

dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);

> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> 
> Stray ":" at the end of the message.
> 

Fixed in next patch.

> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		ret = mtk_cam_video_register_device(cam_dev, i);
> > +		if (ret)
> > +			goto fail_video_register;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_video_register:
> > +	i--;
> 
> This could be moved into the for clause, as the initialization statement.
> 

Fixed in next patch.

> > +	for (; i >= 0; i--) {
> 
> i is unsigned. Did this compile without warnings?
> 
> > +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> > +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> > +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
> 
> Should we move this into mtk_cam_video_unregister_device() to be consistent
> with registration?
> 

Fixed in next patch.

> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> 
> Please print errors only where they actually happen, not at the cleanup.
> 

Fixed in next patch.

> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> 
> nit: Move the declaration inside the for loop, since the variable is only
> used there.
> 

Fixed in next patch.

> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		dev = &cam_dev->vdev_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> 
> This was allocated using devm_kcalloc(), so no need to free it explicitly.
> 

Fixed in next patch.

> > +
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_CIO_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_CIO_PAD_SINK,
> > +				    0);
> > +	if (ret) {
> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> 
> Why not just put the contents of this function inside 
> mtk_cam_dev_notifier_complete()?
> 

Ok, we will mtk_cam_dev_complete() function and move its content into
mtk_cam_dev_notifier_complete().

> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> 
> Should we somehow check that the entity we got is seninf indeed and there
> was no mistake in DT?
> 

How about to check the entity function of seninf device?

if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
	dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
		return -ENODEV;
}

If we need to check DT, may we need to implement this in parse_endpoint
callback function of v4l2_async_notifier_parse_fwnode_endpoints?

> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> 
> bound
> 
> Also please make this dev_dbg().
> 

Fixed in next patch.

> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = NULL;
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> 
> unbound
> 

Fixed in next patch.

> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> > +		NULL);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> 
> Could we print some error messages for the 2 cases above?
> 

Fixed in next patch.

> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> 
> I don't think we need vidioc_*_input. At least the current implementation in
> this patch doesn't seem to do anything useful.
> 
> > +	/* buffer queue management */
> 
> Drop this comment, as it's obvious that the callbacks with "buf" in the name
> are related to buffers, there are some non-buffer callbacks below too (e.g.
> vidioc_subscribe_event) and also the other ops structs below don't have such
> comment.
> 

Fixed in next patch.

> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> 
> PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
> making it 512 * SZ_1K?
> 
> That said, it should normally be just sizeof(struct some_struct_used_here).
> 
> Same for the other entries below.
> 

ok, we will change the size unit from PAGE_SIZE to SZ_1K.
If we finalize the meta structures design, we may change to use sizeof.

> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = IMG_MAX_WIDTH,
> > +			.min_width = IMG_MIN_WIDTH,
> > +			.max_height = IMG_MAX_HEIGHT,
> > +			.min_height = IMG_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = RRZ_MAX_WIDTH,
> > +			.min_width = RRZ_MIN_WIDTH,
> > +			.max_height = RRZ_MAX_HEIGHT,
> > +			.min_height = RRZ_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 1,
> 
> Why not just make it always 0 and move the default format to the beginning
> of stream_out_fmts[]?
> 

Fixed in next patch.

> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[0],
> 
> nit: If you only need to point to these data once, you could define them in
> place, as a compound literal, like
> 

Fixed in next patch.

> >                 .frmsizes = &(struct v4l2_framesizeenum) {
> >                         // initialize here
> >                 },
> 

Ditto

> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[1],
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> 
> link_flags seems to be 0 for all nodes.
> 

Ditto.

> > +		.capture = true,
> 
> We already know this from .buf_type. We can check using the
> V4L2_TYPE_IS_OUTPUT() macro.
> 

Ditto.

> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> 
> I don't think this is correct. The description suggests one specific format
> (local motion vector histogram), so the queue should probably be hardwired
> to that format?
> 

Yes, we will set num_fmts = 1 for meta nodes.

> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> 
> Where does this number come from?
> 

The default maximum VB2 buffer count is 32.
In order to limit memory usage, we like to limit the maximum buffer
counts in the driver layer. The maximum buffer count is estimated
according to our camera MW.

#define VB2_MAX_FRAME	(32)

> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
> 
> < ARRAY_SIZE(output_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
> 
> < ARRAY_SIZE(capture_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> 
> Do we need this additional mtk_cam_dev struct? Couldn't we just use
> mtk_isp_p1 here?
> 

We remove pdev field and add dev field in mtk_cam_dev struct.

> > +	mtk_cam_dev_queue_setup(cam_dev);
> > +	/* v4l2 sub-device registration */
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> 
> This debugging message doesn't seem very useful. Please remove.
> 

Fixed in next patch.

> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret) {
> > +		mtk_cam_v4l2_unregister(cam_dev);
> 
> Please use an error path on the bottom of the function instead. With goto
> labels named after the next clean-up step that needs to be performed.
> 

Fixed in next patch.

> > +		return ret;
> > +	}
> > +
> > +	spin_lock_init(&cam_dev->req_lock);
> > +	INIT_LIST_HEAD(&cam_dev->req_list);
> > +	mutex_init(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> 
> "release" is normally used for file_operations. How about "cleanup"?
> 

Fixed in next patch.

> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	mutex_destroy(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> I'd suggest moving any generic API implementation code (platform_device,
> V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
> level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
> mtk_cam_hw.c.
> 

Fixed in next patch.

> > +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..825cdf20643a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,173 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
> 
> Maybe it's not a critical thing, but generally it's a good practice to just
> explicitly specific this name somewhere, e.g. struct
> platform_driver::driver::name and then just refer to dev_driver_name(). It
> makes it easier to make sure that the name stays the same everywhere.
> 

Fixed in next patch.

> > +
> > +#define MTK_CAM_P1_META_IN_0			0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> 
> Since these are just some utility definitions, we can use enum instead of
> assigning the values by hand.
> 

We will revise these definitions as below.

/* ID enum value for struct mtk_cam_dev_node_desc:id */
enum  {
	MTK_CAM_P1_META_IN_0 = 0,
	MTK_CAM_P1_MAIN_STREAM_OUT,
	MTK_CAM_P1_PACKED_BIN_OUT,
	MTK_CAM_P1_META_OUT_0,
	MTK_CAM_P1_META_OUT_1,
	MTK_CAM_P1_META_OUT_2,
	MTK_CAM_P1_META_OUT_3,
	MTK_CAM_P1_TOTAL_NODES
};

> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> > +#define MTK_CAM_P1_META_OUT_0			3
> > +#define MTK_CAM_P1_META_OUT_1			4
> > +#define MTK_CAM_P1_META_OUT_2			5
> > +#define MTK_CAM_P1_META_OUT_3			6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE		6
> 
> Ditto.
> 

Ditto.

> > +
> > +#define MTK_CAM_P1_TOTAL_NODES			7
> 
> Please just add the two totals together rather than manually specifying the
> value.
> 

Ditto.

> > +
> > +struct mtk_cam_dev_request {
> > +	struct media_request	req;
> > +	struct list_head	list;
> > +};
> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +	unsigned int		node_id;
> > +};
> 
> Could you add kerneldoc comments for the 2 structs?
> 

Fixed in next patch.

> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + * @frmsizes:	supported frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> 
> vdev_pad?
> 
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> 
> Please fix the order of the documentation to match the order of the struct.
> 

Fixed in next patch.

> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct video_device vdev;
> 
> Not documented above.
> 

Fixed in next patch.

> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.
Btw, if we remove this, we will got complain from checkpatch.pl script.

> > +	struct mutex lock;
> > +	/* protect for pending_list */
> 
> It's already documented in the kerneldoc comment.
> 

Ditto.

> > +	spinlock_t slock;
> 
> How about calling it pending_list_lock?
> 

We will rename to buf_list to track all en-queue buffers in this video
node.

struct mtk_cam_video_device {
	unsigned int id;
	unsigned int enabled;
	struct v4l2_format vdev_fmt;
	struct mtk_cam_dev_node_desc desc;
	struct video_device vdev;
	struct media_pad vdev_pad;
	struct vb2_queue vbq;
	struct v4l2_ctrl_handler ctrl_handler;
	struct list_head buf_list;
	struct mutex lock;
	spinlock_t buf_list_lock;
};

> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @lock:       The mutex protecting video device open/release operations
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> > + * @req_list:	Lins to keep media requests before streaming on
> > + * @req_lock:	Protect the req_list data
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> 
> This probably isn't the best place for graph topology description. I think
> we actually want a separate documentation file for this, similar to
> Documentation/media/v4l-drivers/ipu3.rst.
> 

Ok, we will drop our graph topology comment & discuss how to come out
another separate document.

> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	/* protect video device open/release operations */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	struct mutex lock;
> > +	unsigned int streaming:1;
> > +	atomic_t streamed_node_count;
> > +	struct list_head req_list;
> > +	/* protect for req_list */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	spinlock_t req_lock;
> 
> How about calling it req_list_lock?
> 

Below is new mtk_cam_dev structure.
We will use job to handle request.

struct mtk_cam_dev {
	struct device *dev;
	struct device *smem_dev;
	struct media_pipeline pipeline;
	struct media_device media_dev;
	struct v4l2_subdev subdev;
	struct v4l2_device v4l2_dev;
	struct v4l2_async_notifier notifier;
	struct media_pad *subdev_pads;
	struct v4l2_ctrl_handler ctrl_handler;
	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
	struct v4l2_subdev *seninf;
	struct v4l2_subdev *sensor;
	struct mutex lock;
	unsigned int streaming:1;
	unsigned int enabled_dmas;
	unsigned int enabled_node_count;
	atomic_t streamed_node_count;
	struct list_head pending_job_list;
	spinlock_t pending_job_lock;
	struct list_head running_job_list;
	spinlock_t running_job_lock;
	atomic_t running_job_count;
};


> Best regards,
> Tomasz
> 

Thanks again for your many inputs on this patch.
It is helpful for us.

Best regards,

Jungo




^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-18  4:39         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-18  4:39 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi, Tomasz:

On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > Implement standard V4L2 video driver that utilizes V4L2
> > and media framework APIs. In this driver, supports one media
> > device, one sub-device and seven video devices during
> > initialization. Moreover, it also connects with sensor and
> > seninf drivers with V4L2 async APIs.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "media: support Mediatek sensor interface driver"[1].
> > 
> > ISP P1 sub-device communicates with seninf sub-device with CIO.
> > 
> > [1]. media: support Mediatek sensor interface driver
> > https://patchwork.kernel.org/cover/10979135/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c    | 1674 +++++++++++++++++
> >  .../mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h    |  173 ++
> >  3 files changed, 1848 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> > 
> 
> Thanks for the patch. Please see my comments inline.
> 
> [snip]
> 

Appreciate your comments on this patch.
Please check my replied inline.

> > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > +				      struct media_request *new_req)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > +
> > +	if (!cam_dev->streaming) {
> > +		cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > +		spin_lock(&cam_dev->req_lock);
> > +		list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > +		spin_unlock(&cam_dev->req_lock);
> > +		dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > +		return;
> > +	}
> > +
> > +	/* Normal enqueue flow */
> > +	if (new_req) {
> > +		mtk_isp_req_enqueue(dev, new_req);
> > +		return;
> > +	}
> > +
> > +	/* Flush all media requests wehen first stream on */
> > +	list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > +		list_del(&req->list);
> > +		mtk_isp_req_enqueue(dev, &req->req);
> > +	}
> > +}
> 
> This will have to be redone, as per the other suggestions, but generally one
> would have a function that tries to queue as much as possible from a list to
> the hardware and another function that adds a request to the list and calls
> the first function.
> 

We revised this function as below.
First to check the en-queue conditions:
a. stream on 
b. The composer buffers in SCP are 3, so we only could has 3 jobs
at the same time.


Second, try to en-queue the frames in the pending job if possible and
move them into running job list if possible.

The request has been inserted into pending job in mtk_cam_req_validate
which is used to validate media_request.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam_dev->running_job_count);
	unsigned long flags;

	if (!cam_dev->streaming ||
	    buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam_dev->pending_job_lock);
	list_for_each_entry_safe(req, req_prev,
				 &cam_dev->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam_dev->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam_dev->pending_job_lock);

	list_for_each_entry_safe(req, req_prev,
				 &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam_dev->running_job_lock, flags);
		list_add_tail(&req->list, &cam_dev->running_job_list);
		spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);

		mtk_isp_req_enqueue(cam_dev, req);
	}
}


> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_mdev_to_dev(req->mdev);
> > +
> > +	vb2_request_queue(req);
> > +	mtk_cam_req_try_isp_queue(cam_dev, req);
> 
> Looks like this driver is suffering from versy similar problems in request
> handling as the DIP driver used to.
> 
> I'd prefer to save my time and avoid repeating the same comments, so please
> check my comments for the DIP driver and apply them to this one too:
> 
> https://patchwork.kernel.org/patch/10905223/
> 

Yes, we will follow the same design of DIP and replace this function by
vb2_request_queue and defined new request structure. 

/*
 * struct mtk_cam_dev_request - MTK camera device request.
 *
 * @req: Embedded struct media request.
 * @frame_params: The frame info. & address info. of enabled DMA nodes.
 * @frame_work: work queue entry for frame transmission to SCP.
 * @list: List entry of the object for @struct mtk_cam_dev:
 *        pending_job_list or running_job_list.
 * @buf_count: Buffer count in this request.
 *
 */
struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};


> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static __u32 img_get_pixel_byte_by_fmt(__u32 pix_fmt)
> 
> Doesn't this function return bits not bytes?
> 

Yes, the unit sould be bits, not bytes.
We will rename this function to get_pixel_bits.

static unsigned int get_pixel_bits(unsigned int pix_fmt)

> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static __u32 img_cal_main_stream_stride(struct device *dev, __u32 width,
> > +					__u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> 
> The __ prefixed types should be used only inside UAPI. Please change the
> driver to use the normal ones.
> 

Ok, we will fix our usage in our driver source code.

> > +
> > +	width = ALIGN(width, 4);
> 
> If there is some alignment requirement for width, it should be handled by
> TRY_/S_FMT and here we should already assume everything properly aligned.
> 

We will follow your suggestion to move this code login in TRY_/S_FMT
functions.

> > +	stride = ALIGN(DIV_ROUND_UP(width * pixel_byte, 8), 2);
> > +
> > +	dev_dbg(dev, "main width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_packed_out_stride(struct device *dev, __u32 width,
> > +				       __u32 pix_fmt)
> > +{
> > +	__u32 stride;
> > +	__u32 pixel_byte = img_get_pixel_byte_by_fmt(pix_fmt);
> > +
> > +	width = ALIGN(width, 4);
> 
> Ditto.
> 

Will fix.

> > +	stride = DIV_ROUND_UP(width * 3, 2);
> 
> Could we introduce a local variable for this intermediate value, so that its
> name could explain what the value is?
> 
> > +	stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > +
> > +	if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > +		stride = ALIGN(stride, 4);
> 
> Is it expected that only the F10 format needs this alignment?
> 

yes, if the pixel bits of image format is 10, the byte alignment of bpl
should be 4. Otherwise, it is 8. We will revise this and add more
comments.

/* 4 bytes alignment for 10 bit other are 8 bytes alignment */
	if (pixel_bits == 10)
		bpl = ALIGN(bpl, 4);
	else
		bpl = ALIGN(bpl, 8);

> > +
> > +	dev_dbg(dev, "packed width:%d, stride:%d\n", width, stride);
> > +
> > +	return stride;
> > +}
> > +
> > +static __u32 img_cal_stride(struct device *dev,
> > +			    int node_id,
> > +			    __u32 width,
> > +			    __u32 pix_fmt)
> > +{
> > +	__u32 bpl;
> > +
> > +	/* Currently, only support one_pixel_mode */
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT)
> > +		bpl = img_cal_main_stream_stride(dev, width, pix_fmt);
> > +	else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT)
> > +		bpl = img_cal_packed_out_stride(dev, width, pix_fmt);
> > +
> > +	/* For DIP HW constrained, it needs 4 byte alignment */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	return bpl;
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	unsigned int i;
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Calcuate mplane pix format */
> > +static void
> > +mtk_cam_dev_cal_mplane_fmt(struct device *dev,
> > +			   struct v4l2_pix_format_mplane *dest_fmt,
> > +			   unsigned int node_id)
> > +{
> > +	unsigned int i;
> > +	__u32 bpl, sizeimage, imagsize;
> 
> Perhaps s/sizeimage/plane_size/ and s/imagsize/total_size/?
> 

Has fixed it.
Btw we will only support 1 plane, there is no need for plane_size.

> > +
> > +	imagsize = 0;
> > +	for (i = 0 ; i < dest_fmt->num_planes; ++i) {
> > +		bpl = img_cal_stride(dev,
> > +				     node_id,
> > +				     dest_fmt->width,
> > +				     dest_fmt->pixelformat);
> > +		sizeimage = bpl * dest_fmt->height;
> > +		imagsize += sizeimage;
> > +		dest_fmt->plane_fmt[i].bytesperline = bpl;
> > +		dest_fmt->plane_fmt[i].sizeimage = sizeimage;
> > +		memset(dest_fmt->plane_fmt[i].reserved,
> > +		       0, sizeof(dest_fmt->plane_fmt[i].reserved));
> 
> This memset is not needed. The core clears the reserved fields
> automatically:
> 
> https://elixir.bootlin.com/linux/v5.2/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1559
> 
> (We may want to backport the patch that added that to our 4.19 branch.)
> 

Ok, we will remove both memset functions in this function.


> > +		dev_dbg(dev, "plane:%d,bpl:%d,sizeimage:%u\n",
> > +			i,  bpl, dest_fmt->plane_fmt[i].sizeimage);
> > +	}
> > +
> > +	if (dest_fmt->num_planes == 1)
> > +		dest_fmt->plane_fmt[0].sizeimage = imagsize;
> 
> Hmm, we only seem to support 1 plane raw formats in this driver. Does the
> hardware support any formats with more than 1 plane? If not, all the code
> should be simplified to just assume 1 plane.
> 

No, MTK P1 ISP HW only supports raw formats with 1 plane.
We will revise our source codes to support only 1 plane.

> > +}
> > +
> > +static void
> > +mtk_cam_dev_set_img_fmt(struct device *dev,
> > +			struct v4l2_pix_format_mplane *dest_fmt,
> > +			const struct v4l2_pix_format_mplane *src_fmt,
> > +			unsigned int node_id)
> > +{
> > +	dest_fmt->width = src_fmt->width;
> > +	dest_fmt->height = src_fmt->height;
> > +	dest_fmt->pixelformat = src_fmt->pixelformat;
> > +	dest_fmt->field = src_fmt->field;
> > +	dest_fmt->colorspace = src_fmt->colorspace;
> > +	dest_fmt->num_planes = src_fmt->num_planes;
> > +	/* Use default */
> > +	dest_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest_fmt->xfer_func =
> > +		V4L2_MAP_XFER_FUNC_DEFAULT(dest_fmt->colorspace);
> > +	memset(dest_fmt->reserved, 0, sizeof(dest_fmt->reserved));
> 
> Given that src_fmt should already be validated and have any fields adjusted
> to match the driver requirements, wouldn't all the lines above be equivalent
> to *dest_fmt = *src_fmt?
> 
> We probably want to move setting all the constant fields to
> mtk_cam_vidioc_try_fmt().
> 

Ok, we will remove mtk_cam_dev_set_img_fmt function
and use *dest_fmt = *src_fmt directly in the caller.
Moreover, all the constant fields are moved to mtk_cam_vidioc_try_fmt()
function.

> > +
> > +	dev_dbg(dev, "%s: Dest Fmt:%c%c%c%c, w*h:%d*%d\n",
> > +		__func__,
> > +		(dest_fmt->pixelformat & 0xFF),
> > +		(dest_fmt->pixelformat >> 8) & 0xFF,
> > +		(dest_fmt->pixelformat >> 16) & 0xFF,
> > +		(dest_fmt->pixelformat >> 24) & 0xFF,
> > +		dest_fmt->width,
> > +		dest_fmt->height);
> > +
> > +	mtk_cam_dev_cal_mplane_fmt(dev, dest_fmt, node_id);
> 
> This should have been called already before this function was called,
> because src_fmt should be already expected to contain valid settings. In
> fact, this is already called in mtk_cam_vidioc_try_fmt().
> 

Ok, we will revise this.

> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct device *dev,
> 
> Please don't pass struct device pointer around, but instead just the main
> driver data struct, which should be much more convenient for accessing
> various driver data. Please fix the other functions as well.
> 

Ok, we will revise this coding style in our source codes.

> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (queue_desc->image) {
> > +		mtk_cam_dev_set_img_fmt(dev,
> > +					&dest->fmt.pix_mp,
> > +					&default_fmt->fmt.pix_mp,
> > +					queue_desc->id);
> 
> We should probably just call mtk_cam_vidioc_s_fmt() here, with a dummy
> v4l2_format struct and have any incorrect fields replaced by
> mtk_cam_vidioc_try_fmt(), since it's the same logic, as if setting invalid
> v4l2_format at runtime.
> 

Ok, we will revise this.

> > +	} else {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +	}
> > +}
> > +
> > +static int mtk_cam_isp_open(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	int ret;
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	ret = v4l2_fh_open(file);
> > +	if (ret)
> > +		goto unlock;
> > +
> > +	ret = v4l2_pipeline_pm_use(&node->vdev.entity, 1);
> 
> Please don't power on open. Normally applications keep the device nodes open
> all the time, so they would keep everything powered on.
> 
> Normally this should be done as late as possible, ideally when starting the
> streaming.
> 

Ok, we will remove this function and just call 4l2_fh_open(file)
function.

> > +	if (ret)
> > +		dev_err(dev, "%s fail:%d", __func__, ret);
> > +
> > +unlock:
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_isp_release(struct file *file)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	mutex_lock(&cam_dev->lock);
> > +	v4l2_pipeline_pm_use(&node->vdev.entity, 0);
> > +	vb2_fop_release(file);
> > +	mutex_unlock(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> If we remove power handling from open and release, we should be able to just
> use v4l2_fh_open() and vb2_fop_release() directly in the
> v4l2_file_operations struct.
> 

Ok, we will fix this.

> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > +	struct media_entity *entity;
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct v4l2_subdev *sensor;
> 
> This variable would be unitialized if there is no streaming sensor. Was
> there no compiler warning generated for this?
> 

No, there is no compiler warning.
But, we will assign sensor to NULL to avoid unnecessary compiler warning
with different compiler options.

> > +
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x\n",
> > +			entity->name, entity->function);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "Sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "Sensor is not connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	/* Align vb2_core_streamon design */
> > +	if (cam_dev->streaming) {
> > +		dev_warn(dev, "already streaming\n", dev);
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ok, we will move this logic check in the mtk_cam_sd_s_stream function.

> > +
> > +	if (!cam_dev->seninf) {
> > +		dev_err(dev, "no seninf connected:%d\n", ret);
> > +		return -EPERM;
> 
> I don't think -EPERM is a good error code here. It's about a missing seninf
> device, so perhaps -ENODEV?
> 

Fix it in next patch.

> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam_dev->sensor = mtk_cam_cio_get_active_sensor(cam_dev);
> > +	if (!cam_dev->sensor)
> > +		return -EPERM;
> 
> > -ENODEV
> 
> > +
> > +	ret = mtk_isp_config(dev);
> > +	if (ret)
> > +		return -EPERM;
> 
> Maybe just return ret?
> 

Fix it in next patch.

> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream on failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		goto fail_sensor_on;
> > +	}
> > +
> > +	cam_dev->streaming = true;
> > +	mtk_cam_req_try_isp_queue(cam_dev, NULL);
> > +	isp_composer_stream(dev, 1);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_on:
> > +	v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +
> > +	return -EPERM;
> 
> return ret?
> 

Fix it in next patch.

> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	if (!cam_dev->streaming) {
> > +		dev_warn(dev, "already stream off");
> > +		return 0;
> > +	}
> 
> Could we check this in the caller?
> 

Ditto.

> > +
> > +	ret = v4l2_subdev_call(cam_dev->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam_dev->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "%s stream off failed:%d\n",
> > +			cam_dev->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	isp_composer_stream(dev, 0);
> 
> Shouldn't we synchronously wait for the streaming to stop here? Otherwise we
> can't guarantee that the hardware releases all the memory that we're going
> to free once this function returns.
> 

We will add  below functions.
1. Stream off ISP HW
2. Stop ISP HW
3. Clear all pending & running request lists.

	cam_dev->streaming = false;
	mtk_isp_stream(cam_dev, 0);
	mtk_isp_hw_release(cam_dev);
	mtk_cam_dev_req_clear(cam_dev);

	dev_dbg(dev, "streamed off Pass 1\n");

> > +	cam_dev->streaming = false;
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	if (enable)
> > +		return mtk_cam_cio_stream_on(cam_dev);
> > +	else
> > +		return mtk_cam_cio_stream_off(cam_dev);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_sd_s_power(struct v4l2_subdev *sd, int on)
> > +{
> > +	struct mtk_cam_dev *cam_dev = mtk_cam_subdev_to_dev(sd);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s:%d", __func__, on);
> > +
> > +	return on ? mtk_isp_power_init(cam_dev) :
> > +		    mtk_isp_power_release(&cam_dev->pdev->dev);
> 
> s_power is a historical thing and we shouldn't be implementing it. Instead,
> we should use runtime PM and call pm_runtime_get_sync(), pm_runtime_put()
> whenever we start and stop streaming respectively.
> 

Ok, we will remove this callback function.

> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> 
> I assume this check is needed, because the pads with higher indexes are not
> video nodes? If so, a comment would be helpful here.
> 

Yes, we will new comment as below.

	/*
	 * Check video nodes is enabled by link setup.
	 * The pad index of video node should be less than       
         * MTK_CAM_P1_TOTAL_NODES.
	 */
	if (pad < MTK_CAM_P1_TOTAL_NODES)
		cam_dev->vdev_nodes[pad].enabled =
			!!(flags & MEDIA_LNK_FL_ENABLED);

> > +		cam_dev->vdev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *mtk_cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = &mtk_cam_dev->pdev->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> 
> This can be folded into the declaration.
> 

Fix it in next patch.

> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > +		__func__,
> > +		node->id,
> > +		buf->vbb.request_fd,
> > +		buf->vbb.vb2_buf.index);
> > +
> > +	/* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > +	if (vb->vb2_queue->uses_requests)
> > +		return;
> 
> I'd suggest removing non-request support from this driver. Even if we end up
> with a need to provide compatibility for non-request mode, then it should be
> built on top of the requests mode, so that the driver itself doesn't have to
> deal with two modes.
> 

The purpose of non-request function in this driver is needed by
our camera middle-ware design. It needs 3A statistics buffers before
image buffers en-queue. So we need to en-queue 3A statistics with
non-request mode in this driver. After MW got the 3A statistics data, it
will en-queue the images, tuning buffer and other meta buffers with
request mode. Based on this requirement, do you have any suggestion?
For upstream driver, should we only consider request mode?

> > +
> > +	/* Added the buffer into the tracking list */
> > +	spin_lock(&node->slock);
> > +	list_add_tail(&buf->list, &node->pending_list);
> > +	spin_unlock(&node->slock);
> > +
> > +	mtk_isp_enqueue(dev, node->desc.dma_port, buf);
> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *smem_dev = cam_dev->smem_dev;
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(&buf->vbb.vb2_buf, 0);
> > +	buf->scp_addr = 0;
> 
> Just a reminder that this will have to be reworked according to my comments
> for the memory allocation patch.
> 

Yes, we have revised this implementation according to the review of
below patch set.

https://patchwork.kernel.org/patch/10985833/

> > +
> > +	/* scp address is only valid for meta input buffer */
> > +	if (node->desc.smem_alloc)
> > +		buf->scp_addr = mtk_cam_smem_iova_to_scp_addr(smem_dev,
> > +							      buf->daddr);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size)
> > +		return -EINVAL;
> 
> For OUTPUT buffers we need to check if vb2_get_plane_payload() == size.
> Otherwise we could get not enough or invalid data.
> 

Fixed in next patch.

> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> 
> This shouldn't be called on OUTPUT buffers.
> 

Ditto.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> 
> We should also verify that *num_planes == 1, as we don't support more
> planes in this driver.
> 

Ok, here is new check logic.

	if (*num_planes) {
		if (sizes[0] < size || *num_planes != 1)
			return -EINVAL;
	} else {
		*num_planes = 1;
		sizes[0] = size;
	}


> > +		if (sizes[0] < size)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam_dev,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct mtk_cam_dev_request *req, *req0;
> > +	struct media_request_object *obj, *obj0;
> > +	struct vb2_buffer *vb;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: node:%s", __func__, node->vdev.name);
> > +
> > +	/* Return all buffers */
> > +	spin_lock(&node->slock);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> 
> nit: One would normally call the second argument "prev", or "b_prev".
> 

Ok, we will follow this coding convention in our source codes. 

> > +		vb = &b->vbb.vb2_buf;
> > +		if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> We shouldn't need to check the buffer state.
> 

Fixed in next patch.

> > +			vb2_buffer_done(vb, state);
> > +		list_del(&b->list);
> > +	}
> > +	spin_unlock(&node->slock);
> > +
> > +	spin_lock(&cam_dev->req_lock);
> > +	list_for_each_entry_safe(req, req0, &cam_dev->req_list, list) {
> 
> nit: Ditto.
> 

Fixed in next patch.

> > +		list_for_each_entry_safe(obj, obj0, &req->req.objects, list) {
> 
> Need to check if the object is a buffer.
> 

Fixed in next patch.

> > +			vb = container_of(obj, struct vb2_buffer, req_obj);
> > +			if (vb->state == VB2_BUF_STATE_ACTIVE)
> 
> vb->state shouldn't be accessed directly from the drivers.
> 

nit: Ditto.

> Generally, the need to check the state here would suggest that there is
> something wrong with how the driver manages the requests. The list that is
> being iterated here shouldn't contain any requests that have buffers that
> aren't active. That will be achieved if my comments for the request handling
> in the DIP driver are applied to this driver as well.
> 
> > +				vb2_buffer_done(vb, state);
> > +		}
> > +		list_del(&req->list);
> > +	}
> > +	spin_unlock(&cam_dev->req_lock);
> > +
> > +	if (node->vbq.uses_requests)
> > +		mtk_isp_req_flush_buffers(&cam_dev->pdev->dev);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	unsigned int node_count = cam_dev->subdev.entity.use_count;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> 
> How is this synchronized with mtk_cam_media_link_setup()?
> 

We will follow your suggestion and below is our proposal for this
function.

1. Use !cam_dev->pipeline.streaming_count to decide the first node to
stream-on.
2.a If yes, do the following steps
    2.a-1 Call media_pipeline_start function to prevent the link
configuration changes.
    2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
video nodes are enabled and save it into cam_dev->enabled_node_count.
    2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
user has called stream-on API
2.b jump step 3.

3. Use cam_dev->streamed_node_count to track how many video nodes are
streamed by user space.
4. Check all enabled video nodes are streamed or not based on
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
5. If yes, call s_stream on for P1 sub-device

Do you think it is reasonable?

> > +		dev_err(dev, "Node:%d is not enable\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_no_link;
> > +	}
> > +
> > +	dev_dbg(dev, "%s: count info:%d:%d", __func__,
> > +		atomic_read(&cam_dev->streamed_node_count), node_count);
> > +
> > +	if (atomic_inc_return(&cam_dev->streamed_node_count) < node_count)
> > +		return 0;
> 
> How do we guarantee that cam_dev->subdev.entity.use_count doesn't change
> between calls to this function on different video nodes?
> 

Ditto.

> > +
> > +	/* Start streaming of the whole pipeline now */
> > +	ret = media_pipeline_start(&node->vdev.entity, &cam_dev->pipeline);
> > +	if (ret) {
> > +		dev_err(dev, "%s: Node:%d failed\n", __func__, node->id);
> > +		goto fail_start_pipeline;
> > +	}
> > +
> 
> Related to the above comment: If we start the media pipeline when we start
> streaming on the first node, we would naturally prevent the link
> configuration changes until the last node stops streaming (as long as the
> link is not DYNAMIC). Note that it would only mark the entities as
> streaming, but it wouldn't call their s_stream, which I believe is exactly
> what we would need to solve the problem above.
> 

Ditto.

> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "Node:%d s_stream on failed:%d\n", node->id, ret);
> > +		goto fail_stream_on;
> > +	}
> > +
> > +	return 0;
> > +
> > +fail_stream_on:
> > +	media_pipeline_stop(&node->vdev.entity);
> > +fail_start_pipeline:
> > +	atomic_dec(&cam_dev->streamed_node_count);
> > +fail_no_link:
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +
> > +	if (!node->enabled)
> > +		return;
> 
> It shouldn't be possible for this to happen, because nobody could have
> called start_streaming on a disabled node.
> 

Will remove in next patch.

> > +
> > +	mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> 
> Shouldn't we stop streaming first, so that the hardware operation is
> cancelled and any buffers owned by the hardware are released?
> 

For this function, below is the new code flow.

1. Check the first node to stream off based on 
cam_dev->streamed_node_count & cam_dev->enabled_node_count.
2. If yes, call all s_stream off for P1 sub-device
3. Call mtk_cam_vb2_return_all_buffers for each node
4. Check the last node to stream off
5. If yes, call media_pipeline_stop to allow user space
to perform link configuration changes, such as disable link.

But, for step 5, is it too late for end user to disable link?
For example, for first node, it has called stream off but
can't call disable link until the last node is stream off?

> > +
> > +	dev_dbg(dev, "%s: count info:%d", __func__,
> > +		cam_dev->subdev.entity.stream_count);
> > +
> > +	/* Check the first node to stream-off */
> > +	if (!cam_dev->subdev.entity.stream_count)
> > +		return;
> > +
> > +	media_pipeline_stop(&node->vdev.entity);
> > +
> > +	if (v4l2_subdev_call(&cam_dev->subdev, video, s_stream, 0))
> > +		dev_err(dev, "failed to stop streaming\n");
> > +}
> > +
> > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	v4l2_ctrl_request_complete(vb->req_obj.req,
> > +				   dev->v4l2_dev.ctrl_handler);
> 
> This would end up being called multiple times, once for each video node.
> Instead, this should be called explicitly by the driver when it completed
> the request - perhaps in the frame completion handler?
> 
> With that, we probably wouldn't even need this callback.
> 

First, if we don't implement this callback function, we will receive
kernel warning as below.

https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420

Second, this function is only be called in __vb2_queue_cancel function.
Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
So could we just implement dummy empty function?

 * @buf_request_complete: a buffer that was never queued to the driver
but is
 *			associated with a queued request was canceled.
 *			The driver will have to mark associated objects in the
 *			request as completed; required if requests are
 *			supported.


> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, MTK_CAM_DEV_P1_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, MTK_CAM_DEV_P1_NAME, sizeof(cap->card));
> 
> We could just use dev_driver_name(cam_dev->dev) for both.
> 
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam_dev->media_dev.dev));
> 
> We should just store dev in cam_dev.
> 

Will fix in next patch.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> 
> Is the set of formats available always the same regardless of the sensor
> format?
> 

Yes, ISP P1 HW output formats are always available without impact
by sensor formats. 

> > +	f->flags = 0;
> 
> We need f->description too.
> 

For this description, do you suggest 1). we fill this field in this
function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?

https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152

Basically, we prefer method 1.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (!node->desc.num_fmts)
> > +		return -EINVAL;
> 
> When would that condition happen?
> 

Will remove this in next patch.

> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *in_fmt)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	const struct v4l2_format *dev_fmt;
> > +	__u32  width, height;
> 
> Don't use __ types in implementation, they are here for UAPI purposes. There
> is u32, which you could use instead, but for width and height you don't need
> explicit size, so unsigned int should be good.
> 

Ok, we will revise this in next patch.

> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > +		__func__,
> > +		(in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > +		(in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > +		in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > +
> > +	width = in_fmt->fmt.pix_mp.width;
> > +	height = in_fmt->fmt.pix_mp.height;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > +				       in_fmt->fmt.pix_mp.pixelformat);
> > +	if (dev_fmt) {
> > +		mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +					&in_fmt->fmt.pix_mp,
> > +					&dev_fmt->fmt.pix_mp,
> > +					node->id);
> > +	} else {
> > +		mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +					     &node->desc, in_fmt);
> 
> We shouldn't just load a default format. This function should validate all
> the fields one by one and adjust them to something appropriate.
> 

For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
a. width
b. height
c. pixelformat
d. plane_fmt
    - sizeimage
    - bytesperline
e. num_planes
Other fields are consider constant.

So if the user space passes one pixel format with un-supported, we will
apply the default format firstly and adjust width, height, sizeimage,
and bytesperline. We will focus on validate width & height.
Is it ok?

> > +	}
> 
> CodingStyle: No braces if both if and else bodies have only 1 statement
> each.
> 

Will fix coding style in next patch.

> > +	in_fmt->fmt.pix_mp.width = clamp_t(u32,
> > +					   width,
> > +					   CAM_MIN_WIDTH,
> > +					   in_fmt->fmt.pix_mp.width);
> 
> Shouldn't we clamp this with some maximum value too?
> 

Ok, will revise as below:

	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MAX_HEIGHT, IMG_MIN_HEIGHT);

> > +	in_fmt->fmt.pix_mp.height = clamp_t(u32,
> > +					    height,
> > +					    CAM_MIN_HEIGHT,
> > +					    in_fmt->fmt.pix_mp.height);
> 
> Ditto.
> 

Ditto.

> > +	mtk_cam_dev_cal_mplane_fmt(&cam_dev->pdev->dev,
> > +				   &in_fmt->fmt.pix_mp, node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (cam_dev->streaming)
> > +		return -EBUSY;
> 
> I think this should rather be something like vb2_queue_is_busy(), which
> would prevent format changes if buffers are allocated.
> 

Since vb2_queue_is_busy is static function, would we paste its
implementation in this function to check like this?

	if (node->vdev.queue->owner &&
		node->vdev.queue->owner != file->private_data) {
		dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
		return -EBUSY;
	}

> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +
> > +	/* Configure to video device */
> > +	mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > +				&node->vdev_fmt.fmt.pix_mp,
> > +				&f->fmt.pix_mp,
> > +				node->id);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_input(struct file *file, void *fh,
> > +				     struct v4l2_input *input)
> > +{
> > +	if (input->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(input->name, "camera", sizeof(input->name));
> > +	input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_input(struct file *file, void *fh,
> > +				  unsigned int *input)
> > +{
> > +	*input = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_input(struct file *file,
> > +				  void *fh, unsigned int input)
> > +{
> > +	return input == 0 ? 0 : -EINVAL;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	strscpy(f->description, node->desc.description,
> > +		sizeof(node->desc.description));
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +	.s_power = mtk_cam_sd_s_power,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_ops = {
> 
> nit: mtk_cam_media_entity_ops?
> 

Rename in next patch.

> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_request_complete = mtk_cam_vb2_buf_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = mtk_cam_isp_open,
> > +	.release = mtk_cam_isp_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_req_ops = {
> 
> nit: Those are media ops, so perhaps just mtk_cam_media_ops?
> 

Rename in next patch.

> > +	.link_notify = v4l2_pipeline_link_notify,
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct device *dev,
> > +				  struct media_device *media_dev)
> > +{
> > +	media_dev->dev = dev;
> > +	strscpy(media_dev->model, MTK_CAM_DEV_P1_NAME,
> 
> Could we replace any use of this macro with dev_driver_string(dev) and then
> delete the macro? The less name strings in the driver the better, as there
> is less change for confusing the userspace if few different names are used
> at the same time.
> 

Ok, revised in next patch.

> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_req_ops;
> > +
> > +	return media_device_register(media_dev);
> > +}
> > +
> > +static int mtk_cam_video_register_device(struct mtk_cam_dev *cam_dev, u32 i)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[i];
> 
> Would it make sense to pass node as an argument to this function instead of
> (or in addition to) i?
> 

Ok, revised in next patch.

> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	u32 output = !cam_dev->vdev_nodes[i].desc.capture;
> 
> Why not call it capture instead and avoid the inversion?
> 

Ok, we will revised as below.

unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);

> > +	u32 link_flags = cam_dev->vdev_nodes[i].desc.link_flags;
> > +	int ret;
> > +
> > +	cam_dev->subdev_pads[i].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->enabled = false;
> 
> Are all the nodes optional? If there is any required node, it should be
> always enabled and have the MEDIA_LNK_FL_IMMUTABLE flag set.
> 

Ok, MTK_CAM_P1_MAIN_STREAM_OUT is required node, others are optional.
We will enable TK_CAM_P1_MAIN_STREAM_OUT with MEDIA_LNK_FL_IMMUTABLE |
MEDIA_LNK_FL_ENABLED.

> > +	node->id = i;
> > +	node->vdev_pad.flags = cam_dev->subdev_pads[i].flags;
> 
> Hmm, shouldn't the subdev pads have opposite directions (sink vs source)?
> 

Yes, it is wrong and fix with below statement.

node->vdev_pad.flags = output ? MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;


> > +	mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > +				     &node->desc,
> > +				     &node->vdev_fmt);
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->vdev_fmt.type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam_dev->smem_dev;
> > +	} else {
> > +		vbq->dev = &cam_dev->pdev->dev;
> > +	}
> > +
> > +	if (vbq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		vdev->entity.function =
> > +			MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> 
> This is a video node, so it's just a DMA, not a processing entity. I believe
> all the entities corresponding to video nodes should use MEDIA_ENT_F_IO_V4L.
> 

Ok, it is fixed.

> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +	vbq->min_buffers_needed = 0;	/* Can streamon w/o buffers */
> > +	/* Put the process hub sub device in the vb2 private data */
> 
> What is "process hub" and what "sub device" is this about?
> 

We will drop this comment.

> > +	vbq->drv_priv = cam_dev;
> > +	vbq->lock = &node->lock;
> > +	vbq->supports_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_vb2_queue;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 MTK_CAM_DEV_P1_NAME, node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->lock;
> > +	vdev->v4l2_dev = &cam_dev->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.ops = NULL;
> > +	/* Enable private control for image video devices */
> > +	if (node->desc.image) {
> > +		mtk_cam_ctrl_init(cam_dev, &node->ctrl_handler);
> > +		vdev->ctrl_handler = &node->ctrl_handler;
> > +	}
> > +	video_set_drvdata(vdev, cam_dev);
> > +	dev_dbg(dev, "register vdev:%d:%s\n", i, vdev->name);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vdev;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam_dev->subdev.entity,
> > +					    i, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam_dev->subdev.entity,
> > +					    i, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> > +	if (ret)
> > +		goto fail_link;
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->lock);
> > +	spin_lock_init(&node->slock);
> > +	INIT_LIST_HEAD(&node->pending_list);
> 
> This should be all initialized before registering the video device.
> Otherwise userspace could open the device before these are initialized.
> 

Fixed in next patch.

> > +
> > +	return 0;
> > +
> > +fail_link:
> > +	video_unregister_device(vdev);
> > +fail_vdev:
> > +	vb2_queue_release(vbq);
> > +fail_vb2_queue:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_mem2mem2_v4l2_register(struct mtk_cam_dev *cam_dev)
> 
> This function doesn't have anything to do with mem2mem. How about
> mtk_cam_v4l2_register()?
> 
> Perhaps it would make sense to move any media related code into into
> mtk_cam_media_register(), keep only V4L2 related code here and have the
> caller call the former first and then this one, rather than having such deep
> sequence of nested calls, which makes the driver harder to read.
> 

Fixed in next patch.

> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> 
> How about just storing dev, instead of pdev in the struct? Also, calling the
> argument "cam", would make it as short as cam->dev.
> 

Ok, we will revise this in next patch.

> > +	/* Total pad numbers is video devices + one seninf pad */
> > +	unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	ret = mtk_cam_media_register(dev,
> > +				     &cam_dev->media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +	dev_info(dev, "Register media device: %s, 0x%pK",
> > +		 MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> 
> An info message should be useful to the user in some way. Printing kernel
> pointers isn't useful. Something like "registered media0" could be useful to
> let the user know which media device is associated with this driver if there
> is more than one in the system.
> 

Here is the new log info.

dev_info(dev, "media%d register",cam->media_dev.devnode->minor);


> > +
> > +	/* Set up v4l2 device */
> > +	cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > +	ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_v4l2_dev;
> 
> Please call the labels after the cleanup step that needs to be done. It
> makes it easier to spot any ordering errors.
> 

Will fix in next patch.

> > +	}
> > +	dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> 
> Same as above.
> 

Ditto.

dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);

> > +
> > +	/* Initialize subdev media entity */
> > +	cam_dev->subdev_pads = devm_kcalloc(dev, num_subdev_pads,
> > +					    sizeof(*cam_dev->subdev_pads),
> > +					    GFP_KERNEL);
> > +	if (!cam_dev->subdev_pads) {
> > +		ret = -ENOMEM;
> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam_dev->subdev.entity,
> > +				     num_subdev_pads,
> > +				     cam_dev->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize media pads:%d:\n", ret);
> 
> Stray ":" at the end of the message.
> 

Fixed in next patch.

> > +		goto fail_subdev_pads;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_subdev_pads; i++)
> > +		cam_dev->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam_dev->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam_dev->subdev, &mtk_cam_subdev_ops);
> > +	cam_dev->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_STATISTICS;
> > +	cam_dev->subdev.entity.ops = &mtk_cam_media_ops;
> > +	cam_dev->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam_dev->subdev.name, sizeof(cam_dev->subdev.name),
> > +		 "%s", MTK_CAM_DEV_P1_NAME);
> > +	v4l2_set_subdevdata(&cam_dev->subdev, cam_dev);
> > +
> > +	ret = v4l2_device_register_subdev(&cam_dev->v4l2_dev, &cam_dev->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev:%d\n", ret);
> > +		goto fail_subdev;
> > +	}
> > +	dev_info(dev, "register subdev: %s\n", cam_dev->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		ret = mtk_cam_video_register_device(cam_dev, i);
> > +		if (ret)
> > +			goto fail_video_register;
> > +	}
> > +
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_video_register:
> > +	i--;
> 
> This could be moved into the for clause, as the initialization statement.
> 

Fixed in next patch.

> > +	for (; i >= 0; i--) {
> 
> i is unsigned. Did this compile without warnings?
> 
> > +		video_unregister_device(&cam_dev->vdev_nodes[i].vdev);
> > +		media_entity_cleanup(&cam_dev->vdev_nodes[i].vdev.entity);
> > +		mutex_destroy(&cam_dev->vdev_nodes[i].lock);
> 
> Should we move this into mtk_cam_video_unregister_device() to be consistent
> with registration?
> 

Fixed in next patch.

> > +	}
> > +fail_subdev:
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +fail_subdev_pads:
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +fail_v4l2_dev:
> > +	dev_err(dev, "fail_v4l2_dev mdev: 0x%pK:%d", &cam_dev->media_dev, ret);
> 
> Please print errors only where they actually happen, not at the cleanup.
> 

Fixed in next patch.

> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i;
> > +	struct mtk_cam_video_device *dev;
> 
> nit: Move the declaration inside the for loop, since the variable is only
> used there.
> 

Fixed in next patch.

> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		dev = &cam_dev->vdev_nodes[i];
> > +		video_unregister_device(&dev->vdev);
> > +		media_entity_cleanup(&dev->vdev.entity);
> > +		mutex_destroy(&dev->lock);
> > +		if (dev->desc.image)
> > +			v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > +	}
> > +
> > +	vb2_dma_contig_clear_max_seg_size(&cam_dev->pdev->dev);
> > +
> > +	v4l2_device_unregister_subdev(&cam_dev->subdev);
> > +	media_entity_cleanup(&cam_dev->subdev.entity);
> > +	kfree(cam_dev->subdev_pads);
> 
> This was allocated using devm_kcalloc(), so no need to free it explicitly.
> 

Fixed in next patch.

> > +
> > +	v4l2_device_unregister(&cam_dev->v4l2_dev);
> > +	media_device_unregister(&cam_dev->media_dev);
> > +	media_device_cleanup(&cam_dev->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = media_create_pad_link(&cam_dev->seninf->entity,
> > +				    MTK_CAM_CIO_PAD_SRC,
> > +				    &cam_dev->subdev.entity,
> > +				    MTK_CAM_CIO_PAD_SINK,
> > +				    0);
> > +	if (ret) {
> > +		dev_err(dev, "fail to create pad link %s %s err:%d\n",
> > +			cam_dev->seninf->entity.name,
> > +			cam_dev->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	dev_info(dev, "Complete the v4l2 registration\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam_dev->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> 
> Why not just put the contents of this function inside 
> mtk_cam_dev_notifier_complete()?
> 

Ok, we will mtk_cam_dev_complete() function and move its content into
mtk_cam_dev_notifier_complete().

> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> 
> Should we somehow check that the entity we got is seninf indeed and there
> was no mistake in DT?
> 

How about to check the entity function of seninf device?

if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
	dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
		return -ENODEV;
}

If we need to check DT, may we need to implement this in parse_endpoint
callback function of v4l2_async_notifier_parse_fwnode_endpoints?

> > +	cam_dev->seninf = sd;
> > +	dev_info(&cam_dev->pdev->dev, "%s is bounded\n", sd->entity.name);
> 
> bound
> 
> Also please make this dev_dbg().
> 

Fixed in next patch.

> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam_dev =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam_dev->seninf = NULL;
> > +	dev_dbg(&cam_dev->pdev->dev, "%s is unbounded\n", sd->entity.name);
> 
> unbound
> 

Fixed in next patch.

> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	return mtk_cam_dev_complete(notifier);
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam_dev->notifier, sizeof(struct v4l2_async_subdev),
> > +		NULL);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (!cam_dev->notifier.num_subdevs)
> > +		return -ENODEV;
> 
> Could we print some error messages for the 2 cases above?
> 

Fixed in next patch.

> > +
> > +	cam_dev->notifier.ops = &mtk_cam_async_ops;
> > +	dev_info(&cam_dev->pdev->dev, "mtk_cam v4l2_async_notifier_register\n");
> 
> dev_dbg()
> 

Fixed in next patch.

> > +	ret = v4l2_async_notifier_register(&cam_dev->v4l2_dev,
> > +					   &cam_dev->notifier);
> > +	if (ret) {
> > +		dev_err(&cam_dev->pdev->dev,
> > +			"failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam_dev)
> > +{
> > +	v4l2_async_notifier_unregister(&cam_dev->notifier);
> > +	v4l2_async_notifier_cleanup(&cam_dev->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap_mplane = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_enum_input = mtk_cam_vidioc_enum_input,
> > +	.vidioc_g_input = mtk_cam_vidioc_g_input,
> > +	.vidioc_s_input = mtk_cam_vidioc_s_input,
> 
> I don't think we need vidioc_*_input. At least the current implementation in
> this patch doesn't seem to do anything useful.
> 
> > +	/* buffer queue management */
> 
> Drop this comment, as it's obvious that the callbacks with "buf" in the name
> are related to buffers, there are some non-buffer callbacks below too (e.g.
> vidioc_subscribe_event) and also the other ops structs below don't have such
> comment.
> 

Fixed in next patch.

> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 128 * PAGE_SIZE,
> 
> PAGE_SIZE is a weird unit for specifying generic buffer sizes. How about
> making it 512 * SZ_1K?
> 
> That said, it should normally be just sizeof(struct some_struct_used_here).
> 
> Same for the other entries below.
> 

ok, we will change the size unit from PAGE_SIZE to SZ_1K.
If we finalize the meta structures design, we may change to use sizeof.

> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 300 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 160 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 72 * PAGE_SIZE,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_B14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_SRGB,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F8,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F10,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F12,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = RRZ_MAX_WIDTH,
> > +			.height = RRZ_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_F14,
> > +			.field = V4L2_FIELD_NONE,
> > +			.colorspace = V4L2_COLORSPACE_RAW,
> > +			.num_planes = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_frmsizeenum img_frm_size_nums[] = {
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = IMG_MAX_WIDTH,
> > +			.min_width = IMG_MIN_WIDTH,
> > +			.max_height = IMG_MAX_HEIGHT,
> > +			.min_height = IMG_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +	{
> > +		.index = 0,
> > +		.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +		.stepwise = {
> > +			.max_width = RRZ_MAX_WIDTH,
> > +			.min_width = RRZ_MIN_WIDTH,
> > +			.max_height = RRZ_MAX_HEIGHT,
> > +			.min_height = RRZ_MIN_HEIGHT,
> > +			.step_height = 1,
> > +			.step_width = 1,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[MTK_CAM_P1_TOTAL_OUTPUT] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.description = "ISP tuning parameters",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.capture = false,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[MTK_CAM_P1_TOTAL_CAPTURE] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 1,
> 
> Why not just make it always 0 and move the default format to the beginning
> of stream_out_fmts[]?
> 

Fixed in next patch.

> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[0],
> 
> nit: If you only need to point to these data once, you could define them in
> place, as a compound literal, like
> 

Fixed in next patch.

> >                 .frmsizes = &(struct v4l2_framesizeenum) {
> >                         // initialize here
> >                 },
> 

Ditto

> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 1,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &img_frm_size_nums[1],
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.description = "AE/AWB histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.description = "AF histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.description = "Local contrast enhanced statistics",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.capture = true,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.description = "Local motion vector histogram",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> 
> link_flags seems to be 0 for all nodes.
> 

Ditto.

> > +		.capture = true,
> 
> We already know this from .buf_type. We can check using the
> V4L2_TYPE_IS_OUTPUT() macro.
> 

Ditto.

> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.num_fmts = ARRAY_SIZE(meta_fmts),
> 
> I don't think this is correct. The description suggests one specific format
> (local motion vector histogram), so the queue should probably be hardwired
> to that format?
> 

Yes, we will set num_fmts = 1 for meta nodes.

> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> 
> Where does this number come from?
> 

The default maximum VB2 buffer count is 32.
In order to limit memory usage, we like to limit the maximum buffer
counts in the driver layer. The maximum buffer count is estimated
according to our camera MW.

#define VB2_MAX_FRAME	(32)

> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int i, node_idx;
> > +
> > +	node_idx = 0;
> > +
> > +	/* Setup the output queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_OUTPUT; i++)
> 
> < ARRAY_SIZE(output_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_CAPTURE; i++)
> 
> < ARRAY_SIZE(capture_queues)
> 

Fixed in next patch.

> > +		cam_dev->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev)
> > +{
> > +	int ret;
> > +
> > +	cam_dev->pdev = pdev;
> 
> Do we need this additional mtk_cam_dev struct? Couldn't we just use
> mtk_isp_p1 here?
> 

We remove pdev field and add dev field in mtk_cam_dev struct.

> > +	mtk_cam_dev_queue_setup(cam_dev);
> > +	/* v4l2 sub-device registration */
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "mem2mem2.name: %s\n",
> > +		MTK_CAM_DEV_P1_NAME);
> 
> This debugging message doesn't seem very useful. Please remove.
> 

Fixed in next patch.

> > +	ret = mtk_cam_mem2mem2_v4l2_register(cam_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam_dev);
> > +	if (ret) {
> > +		mtk_cam_v4l2_unregister(cam_dev);
> 
> Please use an error path on the bottom of the function instead. With goto
> labels named after the next clean-up step that needs to be performed.
> 

Fixed in next patch.

> > +		return ret;
> > +	}
> > +
> > +	spin_lock_init(&cam_dev->req_lock);
> > +	INIT_LIST_HEAD(&cam_dev->req_list);
> > +	mutex_init(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_cam_dev_release(struct platform_device *pdev,
> 
> "release" is normally used for file_operations. How about "cleanup"?
> 

Fixed in next patch.

> > +			struct mtk_cam_dev *cam_dev)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam_dev);
> > +	mtk_cam_v4l2_unregister(cam_dev);
> > +
> > +	mutex_destroy(&cam_dev->lock);
> > +
> > +	return 0;
> > +}
> 
> I'd suggest moving any generic API implementation code (platform_device,
> V4L2, VB2, Media Controller, etc.) to mtk_cam.c and then moving any low
> level hardware/firmware-related code from mtk_cam.c and mtk_cam-scp.c to
> mtk_cam_hw.c.
> 

Fixed in next patch.

> > +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> new file mode 100644
> index 000000000000..825cdf20643a
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-v4l2-util.h
> @@ -0,0 +1,173 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_DEV_V4L2_H__
> > +#define __MTK_CAM_DEV_V4L2_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#define MTK_CAM_DEV_P1_NAME			"MTK-ISP-P1-V4L2"
> 
> Maybe it's not a critical thing, but generally it's a good practice to just
> explicitly specific this name somewhere, e.g. struct
> platform_driver::driver::name and then just refer to dev_driver_name(). It
> makes it easier to make sure that the name stays the same everywhere.
> 

Fixed in next patch.

> > +
> > +#define MTK_CAM_P1_META_IN_0			0
> > +#define MTK_CAM_P1_TOTAL_OUTPUT		1
> 
> Since these are just some utility definitions, we can use enum instead of
> assigning the values by hand.
> 

We will revise these definitions as below.

/* ID enum value for struct mtk_cam_dev_node_desc:id */
enum  {
	MTK_CAM_P1_META_IN_0 = 0,
	MTK_CAM_P1_MAIN_STREAM_OUT,
	MTK_CAM_P1_PACKED_BIN_OUT,
	MTK_CAM_P1_META_OUT_0,
	MTK_CAM_P1_META_OUT_1,
	MTK_CAM_P1_META_OUT_2,
	MTK_CAM_P1_META_OUT_3,
	MTK_CAM_P1_TOTAL_NODES
};

> > +
> > +#define MTK_CAM_P1_MAIN_STREAM_OUT		1
> > +#define MTK_CAM_P1_PACKED_BIN_OUT		2
> > +#define MTK_CAM_P1_META_OUT_0			3
> > +#define MTK_CAM_P1_META_OUT_1			4
> > +#define MTK_CAM_P1_META_OUT_2			5
> > +#define MTK_CAM_P1_META_OUT_3			6
> > +#define MTK_CAM_P1_TOTAL_CAPTURE		6
> 
> Ditto.
> 

Ditto.

> > +
> > +#define MTK_CAM_P1_TOTAL_NODES			7
> 
> Please just add the two totals together rather than manually specifying the
> value.
> 

Ditto.

> > +
> > +struct mtk_cam_dev_request {
> > +	struct media_request	req;
> > +	struct list_head	list;
> > +};
> > +
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer	vbb;
> > +	struct list_head	list;
> > +	/* Intenal part */
> > +	dma_addr_t		daddr;
> > +	dma_addr_t		scp_addr;
> > +	unsigned int		node_id;
> > +};
> 
> Could you add kerneldoc comments for the 2 structs?
> 

Fixed in next patch.

> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - node attributes
> > + *
> > + * @id:		 id of the context queue
> > + * @name:	 media entity name
> > + * @description: descritpion of node
> > + * @cap:	 mapped to V4L2 capabilities
> > + * @buf_type:	 mapped to V4L2 buffer type
> > + * @dma_port:	 the dma port associated to the buffer
> > + * @link_flags:	 default media link flags
> > + * @smem_alloc:	 using the cam_smem_drv as alloc ctx or not
> > + * @capture:	 true for capture queue (device to user)
> > + *		 false for output queue (from user to device)
> > + * @image:	 true for image node, false for meta node
> > + * @num_fmts:	 the number of supported formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum V4L2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts:	supported format
> > + * @frmsizes:	supported frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	char *name;
> > +	char *description;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 capture:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure.
> > + *
> > + * @id:		Id for mtk_cam_dev_node_desc or mem2mem2_nodes array
> > + * @enabled:	Indicate the device is enabled or not
> > + * @vdev_fmt:	The V4L2 format of video device
> > + * @vdev_apd:	The media pad graph object of video device
> 
> vdev_pad?
> 
> > + * @vbq:	A videobuf queue of video device
> > + * @desc:	The node attributes of video device
> > + * @ctrl_handler:	The control handler of video device
> > + * @pending_list:	List for pending buffers before enqueuing into driver
> > + * @lock:	Serializes vb2 queue and video device operations.
> > + * @slock:	Protect for pending_list.
> > + *
> 
> Please fix the order of the documentation to match the order of the struct.
> 

Fixed in next patch.

> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct v4l2_format vdev_fmt;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct video_device vdev;
> 
> Not documented above.
> 

Fixed in next patch.

> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct list_head pending_list;
> > +	/* Used for vbq & vdev */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.
Btw, if we remove this, we will got complain from checkpatch.pl script.

> > +	struct mutex lock;
> > +	/* protect for pending_list */
> 
> It's already documented in the kerneldoc comment.
> 

Ditto.

> > +	spinlock_t slock;
> 
> How about calling it pending_list_lock?
> 

We will rename to buf_list to track all en-queue buffers in this video
node.

struct mtk_cam_video_device {
	unsigned int id;
	unsigned int enabled;
	struct v4l2_format vdev_fmt;
	struct mtk_cam_dev_node_desc desc;
	struct video_device vdev;
	struct media_pad vdev_pad;
	struct vb2_queue vbq;
	struct v4l2_ctrl_handler ctrl_handler;
	struct list_head buf_list;
	struct mutex lock;
	spinlock_t buf_list_lock;
};

> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @pdev:	Pointer to platform device
> > + * @smem_pdev:	Pointer to shared memory platform device
> > + * @pipeline:	Media pipeline information
> > + * @media_dev:	Media device
> > + * @subdev:	The V4L2 sub-device
> > + * @v4l2_dev:	The V4L2 device driver
> > + * @notifier:	The v4l2_device notifier data
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device
> > + * @ctrl_handler: The control handler
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes
> > + * @seninf:	Pointer to the seninf sub-device
> > + * @sensor:	Pointer to the active sensor V4L2 sub-device when streaming on
> > + * @lock:       The mutex protecting video device open/release operations
> > + * @streaming:	Indicate the overall streaming status is on or off
> > + * @streamed_node_count: The number of V4L2 video device nodes are streaming on
> > + * @req_list:	Lins to keep media requests before streaming on
> > + * @req_lock:	Protect the req_list data
> > + *
> > + * Below is the graph topology for Camera IO connection.
> > + * sensor 1 (main) --> sensor IF --> P1 sub-device
> > + * sensor 2 (sub)  -->
> 
> This probably isn't the best place for graph topology description. I think
> we actually want a separate documentation file for this, similar to
> Documentation/media/v4l-drivers/ipu3.rst.
> 

Ok, we will drop our graph topology comment & discuss how to come out
another separate document.

> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct platform_device *pdev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct v4l2_ctrl_handler ctrl_handler;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	/* protect video device open/release operations */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	struct mutex lock;
> > +	unsigned int streaming:1;
> > +	atomic_t streamed_node_count;
> > +	struct list_head req_list;
> > +	/* protect for req_list */
> 
> It's already documented in the kerneldoc comment.
> 

Fixed in next patch.

> > +	spinlock_t req_lock;
> 
> How about calling it req_list_lock?
> 

Below is new mtk_cam_dev structure.
We will use job to handle request.

struct mtk_cam_dev {
	struct device *dev;
	struct device *smem_dev;
	struct media_pipeline pipeline;
	struct media_device media_dev;
	struct v4l2_subdev subdev;
	struct v4l2_device v4l2_dev;
	struct v4l2_async_notifier notifier;
	struct media_pad *subdev_pads;
	struct v4l2_ctrl_handler ctrl_handler;
	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
	struct v4l2_subdev *seninf;
	struct v4l2_subdev *sensor;
	struct mutex lock;
	unsigned int streaming:1;
	unsigned int enabled_dmas;
	unsigned int enabled_node_count;
	atomic_t streamed_node_count;
	struct list_head pending_job_list;
	spinlock_t pending_job_lock;
	struct list_head running_job_list;
	spinlock_t running_job_lock;
	atomic_t running_job_count;
};


> Best regards,
> Tomasz
> 

Thanks again for your many inputs on this patch.
It is helpful for us.

Best regards,

Jungo




_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-07-10  9:56       ` Tomasz Figa
  (?)
@ 2019-07-20  9:58         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-20  9:58 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi, Tomasz:

On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> >  4 files changed, 1457 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thanks for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..9e59a6bfc6b7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,126 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT			BIT(0)
> > +#define CMOS_EN_BIT			BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST			BIT(0)
> > +#define HW_PASS1_DON_ST		BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define SW_PASS1_DON_ST		BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST			BIT(4)
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +#define PSO_DONE_ST			BIT(14)
> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM		( \
> > +					VS_INT_ST |\
> > +					SOF_INT_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					SW_PASS1_DON_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM		( \
> > +					SOF_INT_ST |\
> > +					SW_PASS1_DON_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					VS_INT_ST |\
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					RRZO_ERR_ST |\
> > +					AFO_ERR_ST |\
> > +					IMGO_ERR_ST |\
> > +					AAO_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM		( \
> > +					AAO_DONE_ST |\
> > +					AFO_DONE_ST)
> 
> Could we define the values next to the addresses of registers they apply to?
> Also without the _BIT suffix and with the values prefixed with register
> names. For example:
> 
> #define REG_TG_SEN_MODE		        0x0230
> #define TG_SEN_MODE_CMOS_EN		BIT(0)
> 
> #define REG_TG_VF_CON			0x0234
> #define TG_VF_CON_VFDATA_EN		BIT(0)
> 

Fix in next patch.

> > +
> > +/* Status check */
> > +#define REG_CTL_EN			0x0004
> > +#define REG_CTL_DMA_EN			0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2			0x0018
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_VF_CON			0x0234
> > +
> > +#define REG_IMGO_BASE_ADDR		0x1020
> > +#define REG_RRZO_BASE_ADDR		0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +#define REG_AAO_ERR_STAT		0x1368
> > +#define REG_AFO_ERR_STAT		0x136c
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +#define REG_UFEO_ERR_STAT		0x1374
> > +#define REG_PDO_ERR_STAT		0x1378
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +#define REG_PDI_ERR_STAT		0x138c
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +#define REG_HW_FRAME_NUM		0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX		0x14dc
> > +#define REG_META1_VB2_INDEX		0x151c
> 
> I don't believe these registers are really for VB2 indexes.
> 

MTK P1 ISP HW supports frame header spare registers for each DMA, such
as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
frame information in these ISP registers. In this case, we save META0
VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
spare register. These implementation is designed for non-request 3A
DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as 
REG_META0_VB2_INDEX for easy understanding. Moreover, if we only need to
support request mode, we should remove this here.

cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
cmd_params.meta_frame.enabled_dma = dma_port;
cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;

> > +
> > +/* FBC */
> > +#define REG_AAO_FBC_STATUS		0x013c
> > +#define REG_AFO_FBC_STATUS		0x0134
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..c5a3babed69d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +#include <media/v4l2-event.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> 
> Please move below. Just above the platform_driver struct where it's used.
> 

Fix in next patch.

> > +
> > +/* List of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +};
> 
> Please move inside mtk_isp_probe, as a static const local variable. That
> could also let you shorten the name, to clk_names for example.
> 

Fix in next patch.

> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +					 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> 
> nit: You can just initialize the structure in the declaration.
> 

Fix in next patch.

void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
				  unsigned int frame_seq_no)
{
	struct v4l2_event event = {
		.type = V4L2_EVENT_FRAME_SYNC,
		.u.frame_sync.frame_sequence = frame_seq_no,
	};

	v4l2_event_queue(cam->subdev.devnode, &event);
}

> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +}
> > +
> > +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> > +				   unsigned int request_fd,
> > +				   unsigned int frame_seq_no,
> > +				   struct list_head *list_buf,
> > +				   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	u64    timestamp;
> 
> Too many spaces between u64 and timestamp.
> 

Fix in next patch.

> > +
> > +	if (!cam_dev->streaming)
> > +		return;
> > +
> > +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> > +		__func__, request_fd, frame_seq_no, state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	timestamp = ktime_get_ns();
> > +	list_for_each_entry_safe(buf, b0, list_buf, list) {
> 
> nit: s/b0/buf_prev/
> 

Fix in next patch.

> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = timestamp;
> > +		buf->vbb.sequence = frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> Any buffer that is not active shouldn't be on this list. If it is then it's
> a bug somewhere else in the driver. Could be possibly related to the request
> handling issues I pointed out in another comment.
> 

Fix in next patch.

> > +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct vb2_buffer *vb;
> > +
> > +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> > +		return;
> > +
> > +	spin_lock(&node->slock);
> > +	b = list_first_entry(&node->pending_list,
> > +			     struct mtk_cam_dev_buffer,
> > +			     list);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		vb = &b->vbb.vb2_buf;
> > +		if (!vb->vb2_queue->uses_requests &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vb->timestamp = ktime_get_ns();
> > +			b->vbb.sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +			list_del(&b->list);
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&node->slock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_warn(dev,
> > +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				 framejob->frame_seq_no,
> > +				 atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		} else {
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> 
> dequeue
> 

Fix in next patch.

> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> > +		    event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int addr_offset;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	if (cq_num <= frame_num) {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> > +		return cq_num;
> > +	}
> > +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> > +
> > +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(isp_dev->dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> > +		cq_num, frame_num, addr_offset);
> > +
> > +	return cq_num;
> > +}
> > +
> > +static void irq_handle_notify_event(struct isp_device *isp_dev,
> > +				    unsigned int irq_status,
> > +				    unsigned int dma_status,
> > +				    bool sof_only)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	if (irq_status & VS_INT_ST) {
> > +		/* Notify specific HW events to user space */
> > +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> > +					     isp_dev->current_frame);
> 
> Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
> definition of the V4L2_EVENT_FRAME_SYNC event at
> https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html
> 

Fix in next patch.
We will change to use SOF_INT_ST to avoid misunderstanding.

> > +		dev_dbg(dev,
> > +			"frame sync is sent:%d:%d\n",
> > +			isp_dev->sof_count,
> > +			isp_dev->current_frame);
> > +		if (sof_only)
> > +			return;
> 
> If this function can be called only to perform this block, perhaps it should
> be split into two functions?
> 
> Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
> irq_status? Is it expected that in such case the other part of the function
> is executed?
> 

Ok, we will call mtk_cam_dev_event_frame_sync function when receiving
SOF_INT_ST ISR event in the caller and remove this block.

> > +	}
> > +
> > +	if (irq_status & SW_PASS1_DON_ST) {
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask =
> > +		(irq_status & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask =
> > +		(dma_status & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> 
> I can see that all isp_deque_work() does is returning buffers to vb2. I
> don't think we need this intricate system to do that, as we could just do
> it inside the interrupt handler, in isp_irq_cam() directly.
> 

Ok, we will move all dequeue function in the ISR function and remove
this dequeue thread and related codes.

> > +
> > +	dev_dbg(dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irq_status & INT_ST_MASK_CAM),
> > +		(dma_status & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cam_idx, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned int aao_fbc, afo_fbc;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev.streaming)
> > +		return IRQ_HANDLED;
> 
> This shouldn't be needed. The driver needs to unmask the interrupts in
> hardware registers when it starts streaming and mask them when it stops.
> Note that I mean the P1 hardware registers, not disable_irq(), which
> shouldn't be used.
> 

Fix in next patch.

> > +
> > +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> 
> Hmm, reading vb2 buffer index from hardware registers? Was this hardware
> designed exclusively for V4L2? ;)
> 
> Jokes aside, how does the hardware know V4L2 buffer indexes?
> 

This is explained in the above.

> > +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> > +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> > +		return IRQ_HANDLED;
> 
> Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
> get an interrupt without any unmasked hardware IRQs active in the status,
> that's an error somewhere and we should return IRQ_NONE.
> 

Ok, we will check the IRQ EN register firstly and check any unmasked
IRQs for IRQ_NONE case.

> > +
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +
> > +	/* Sof, done order check */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue frame */
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > +		isp_dev->current_frame = hw_frame_num;
> 
> What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> event?
> 

This is a another spare register for frame sequence number usage.
It comes from struct p1_frame_param:frame_seq_no which is sent by
SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
Is it a better understanding?

Below is our frame request handling in current design.

1. Buffer preparation
- Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
other meta histogram buffers (LCSO/LMVO) into one request.
- Accumulated one unique frame sequence number to each request and send
this request to the SCP composer to compose CQ (Command queue) buffer
via SCP_ISP_FRAME IPI command.
- CQ buffer is frame registers set. If ISP registers should be updated
per frame, these registers are configured in the CQ buffer, such as
frame sequence number, DMA addresses and tuning ISP registers.
- One frame request will be composed into one CQ buffer.Once CQ buffer
is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
its corresponding frame sequence number. Based on this, kernel driver
knows which request is ready to be en-queued and save this with
p1_dev->isp_ctx.composed_frame_id.
- The maximum number of CQ buffers in SCP is 3.

2. Buffer en-queue flow
- In order to configure correct CQ buffer setting before next SQF event,
it is depended on by MTK ISP P1 HW CQ mechanism.
- The basic concept of CQ mechanism is loaded ISP CQ buffer settings
when HW_PASS1_DON_ST is received which means DMA output is done.
- Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
address is used. Otherwise, it will loaded one dummy CQ buffer to
bypass.
- So we will check available CQ buffers by comparing composed frame
sequence number & dequeued frame sequence from ISP HW in SOF event.
- If there are available CQ buffers, update the CQ base address to the
next CQ buffer address based on current de-enqueue frame sequence
number. So MTK ISP P1 HW will load this CQ buffer into HW when
HW_PASS1_DON_ST is triggered which is before the next SOF.
- So in next SOF event, ISP HW starts to output DMA buffers with this
request until request is done.
- But, for the first request, it is loaded into HW manually when
streaming is on for better performance.

3. Buffer de-queue flow
- We will use frame sequence number to decide which request is ready to
de-queue.
- We will save some important register setting from ISP HW when SOF is
received. This is because the ISP HW starts to output the data with the
corresponding settings, especially frame sequence number setting.
- When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
done. So we could call isp_deque_request_frame with frame sequence
number to de-queue frame to VB2
- For AAO/AFO buffers, it has similar design concept. Sometimes, if only
AAO/AFO non-request buffers are en-queued without request buffers at the
same time, there will be no SW P1 done event for AAO/AFO DMA done.
Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
- Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
we may try to send another request to SCP for composing.

Hopefully, my explanation is helpful for better understanding our
implementation. If you still have any questions, please let me know. 

> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irq_status & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > +	}
> 
> The if and else blocks do almost the same things just in different order. Is
> it really expected?
> 

If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
the correct sequence should be handle HW_PASS1_DON_ST firstly to check
any de-queued frame and update the next frame setting later.
Normally, this is a corner case or system performance issue.

Btw, we will revise the above implementation as below.


if (irq_status & SOF_INT_ST)
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
					     dequeue_frame_seq_no);

/* Sof, done order check */
if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
	dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

/* Notify IRQ event and de-enqueue frame */
irq_handle_notify_event(p1_dev, irq_status, dma_status);

/* Update frame settings & CQ address for frame en-queue */
enqueue_frame_seq_no = 0;
if (irq_status & SOF_INT_ST)
	enqueue_frame_seq_no = irq_handle_sof(p1_dev,
					      dequeue_frame_seq_no,
					      meta0_vb2_index,
					      meta1_vb2_index); 

> > +
> > +	if (irq_status & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x\n",
> > +			irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irq_status & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> 
> Please just put that string here, otherwise the reader would have no idea
> what message is being printed here.
> 

Fix in next patch.

> > +			'A' + cam_idx,
> > +			isp_dev->sof_count,
> > +			irq_status,
> > +			dma_status,
> > +			hw_frame_num,
> > +			cq_num,
> > +			aao_fbc,
> > +			afo_fbc);
> 
> nit: No need to put each argument in its own line.
> 

Fix in next patch.

> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> 
> This look-up should be done once in probe(). Only the rproc_boot() should
> happen dynamically.
> 

Fix in next patch.

> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +	mutex_init(&isp_ctx->lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +
> > +	return 0;
> 
> Everything here looks like something that should be done once in probe. I
> also don't see a point of having a separate mtk_isp_p1_ctx struct for the
> data above. It could be just located in p1_dev, at least for now.
> 
> If we end up implementing support for multiple contexts, we could isolate
> per-context data then, once we know what's really per-context. For now,
> let's keep it simple.
> 

Ok, we will remove isp_ctx structure and move some fields into p1_dev or
cam_dev,

> > +}
> > +
> > +static int isp_uninit_context(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	if (isp_ctx->isp_deque_thread.thread) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	mutex_destroy(&isp_ctx->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int enabled_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enabled_dma_ports = 0;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		if (cam_dev->vdev_nodes[i].enabled)
> > +			enabled_dma_ports |=
> > +				cam_dev->vdev_nodes[i].desc.dma_port;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> > +
> > +	return enabled_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return RAW_PXL_ID_R;
> > +	default:
> 
> Could we fail here instead?
> 

Ok, we will return invalid value here, like MTK_CAM_RAW_PXL_ID_UNKNOWN.
We will also check this error in the caller.

> > +		return RAW_PXL_ID_B;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return IMG_FMT_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> 
> Could we fail here instead, so that we don't mask some potential errors?
> 

Ok, we will return MTK_CAM_IMG_FMT_UNKNOWN and check this error in the
caller.

> > +		return 10;
> > +	}
> > +}
> > +
> > +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> > +			   const struct v4l2_format *in_fmt,
> > +			   const struct v4l2_subdev_format *sd_format)
> > +{
> > +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> > +
> > +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> 
> Please group operations on the same field together, i.e. remove the blank
> line above size.stride and add one blank line above size.w.
> 

Fix in next patch.

> > +
> > +	out_fmt->crop.left = 0x0;
> > +	out_fmt->crop.top = 0x0;
> > +
> 
> Remove the blank line.
> 

Fix in next patch.

> > +	out_fmt->crop.width = sd_format->format.width;
> > +	out_fmt->crop.height = sd_format->format.height;
> > +
> > +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> > +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> > +		  "img out:%d:%d in:%d:%d",
> > +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> > +		  out_fmt->crop.width, out_fmt->crop.height);
> 
> We should check this earlier and fail the streaming start if there is a
> mismatch between sensor and video node configuration.
> 

Fix in next patch.

> > +
> > +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> > +		out_fmt->pixel_byte,
> > +		out_fmt->img_fmt);
> > +	dev_dbg(dev,
> > +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = isp_init_context(p1_dev);
> > +	if (ret)
> > +		return ret;
> 
> The above function doesn't really seem to be related to power management.
> Should it be called from subdev stream on?
> 

We will rename this function to mtk_isp_hw_init.
But, it will be called when the first video node is streamed on.
This is because we need to initialize the HW firstly for sub-device
stream-on performance.  We need to send some IPI commands, such as
ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
timing.

> > +
> > +	ret = isp_composer_init(dev);
> > +	if (ret)
> > +		goto composer_err;
> 
> This also doesn't seem to be related to power management.
> 

Ok, we will rename to mtk_isp_hw_init to avoid misunderstanding.

> > +
> > +	pm_runtime_get_sync(dev);
> > +
> > +	/* ISP HW INIT */
> > +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> 
> There is some amount of code handling various values of isp_hw_module in
> this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
> code that can't be tested. Please either add support for all the indexes in
> the driver or simplify all the code to just handle CAM_B.
> 

Ok, we will simplify driver code to support single CAM only.
We will remove isp_hw_module usage in the source code.

> > +	/* Use pure RAW as default HW path */
> > +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > +
> > +	isp_composer_hw_init(dev);
> > +	/* Check enabled DMAs which is configured by media setup */
> > +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> 
> Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> necessary?
> 

Yes, it is necessary for non-request buffers design for Camera 3A video
nodes. For 3A video nodes, we just want to know which 3A video nodes are
enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
format from user space. So we just pass the enabled DMA information from
kernel to SCP only. With 3A enabled DMA information, we could configure
related 3A registers in SCP.

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +
> > +composer_err:
> > +	isp_uninit_context(dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_power_release(struct device *dev)
> > +{
> > +	isp_composer_hw_deinit(dev);
> > +	isp_uninit_context(dev);
> 
> These two don't seem to be related to power either.
> 
> Instead, I don't see anything that could undo the rproc_boot() operation
> here.
> 

We will rename this function to mtk_isp_hw_release.
We will also add rproc_shutdown function call here.

int mtk_isp_hw_release(struct mtk_cam_dev *cam)
{
	struct device *dev = cam->dev;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	isp_composer_hw_deinit(p1_dev);
	pm_runtime_put_sync_autosuspend(dev);
	rproc_shutdown(p1_dev->rproc_handle);

	dev_dbg(dev, "%s done\n", __func__);

	return 0;
}

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	unsigned int enabled_dma_ports;
> > +	struct v4l2_format *img_fmt;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> > +
> > +	/* Sensor config */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +
> 
> Unnecessary blank line.
> 

Fix in next patch.

> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix in next patch.

> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_fmt.format.width, sd_fmt.format.height,
> > +		sd_fmt.format.code, sd_fmt.format.field,
> > +		sd_fmt.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* Fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = 0x1;
> > +	/* Support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> 
> Why hexadecimal numerals? Also, what's the meaning of these values? For
> anything boolean, you could just use true and false as a substitute of 0 and
> 1. For anything that has more values, please define the values using macros.
> 

Fix in next patch.
1. Remove unnecessary hexadecimal number usage
2. Use boolean value if possible
3. Assign non-empty field values.

Below is the fixed version.

	config_param.cfg_in_param.continuous = true;
	/* Fix to one pixel mode in default */
	config_param.cfg_in_param.pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
	config_param.cfg_in_param.raw_pixel_id =
		get_sensor_pixel_id(sd_fmt.format.code);
	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
	if (config_param.cfg_in_param.img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
	    config_param.cfg_in_param.raw_pixel_id ==
MTK_CAM_RAW_PXL_ID_UNKNOWN) {
		dev_err(dev, "Unknown cfg img fmt or unknown raw pixel id\n");
		return -EINVAL;
	}

> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_fmt.format.code);
> > +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
> 
> Move the other crop settings from above to here.
> 

Ditto.

> > +
> > +	config_param.cfg_main_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
> 
> No need for the == R_IMGO part.
> 

Since R_IMGO is mandatory video node, we will remove this checking.

> > +		config_param.cfg_main_param.bypass = 0;
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	config_param.cfg_resize_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
> 
> Ditto.
> 

Fix in next patch as below:

if (enabled_dma_ports & R_RRZO) {
	ret = config_img_fmt(cam,
			     &config_param.cfg_resize_param.output,
			     img_fmt, &sd_fmt);
	if (ret)
		return ret;
} else {
	config_param.cfg_resize_param.bypass = true;
}

> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
> 
> Should image DMAs be masked out of this bitmap?
> 

We will replace struct cfg_meta_param with enabled_dmas integer.
So we can pass all enabled DMA masks to SCP. 

> > +
> > +	isp_composer_hw_config(dev, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> > +		     struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> > +}
> > +
> > +void mtk_isp_req_flush_buffers(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_queue_job *job, *j0;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> > +
> > +	if (!atomic_read(&p1_list->queue_cnt))
> > +		return;
> 
> Do we need this explicit check? The code below wouldn't do anything if there
> isn't anything in the list. IMHO we could even remove queue_cnt completely.
> 

Since we have redesign request en-queue mechanism, this function will be
removed.

> > +
> > +	spin_lock(&p1_list->lock);
> > +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
> 
> nit: s/j0/job_prev/
> 

Will apply this naming rule in next patch.

> > +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
> 
> nit: s/b0/buf_pref/
> 

Ditto.

> Also, we should be able to replace this with iterating over the generic list
> of request objects, rather than this internal list.
> 
> > +			list_del(&buf->list);
> > +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> It shouldn't be possible to happen. If you see this check failing, that
> means a problem somewhere else in the driver.
> 

Fix in next patch.

> > +				vb2_buffer_done(&buf->vbb.vb2_buf,
> > +						VB2_BUF_STATE_ERROR);
> > +		}
> > +		list_del(&job->list_entry);
> > +		atomic_dec(&p1_list->queue_cnt);
> > +		kfree(job);
> > +	}
> > +	spin_unlock(&p1_list->lock);
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

We got your point.
Below is new function declaration.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct media_request_object *obj, *obj_safe;
> > +	struct vb2_buffer *vb;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> 
> This allocation shouldn't be needed. The structure should be already a part
> of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> even say that the contents of this struct should be just moved to that one
> to avoid overabstracting.
> 

For this function, we will apply the new design from P2 driver's
comment. Here is the new implementation.

struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

> > +	memset(framejob, 0, sizeof(*framejob));
> 
> Putting the above comment aside, kzalloc() already zeroes the memory.
> 

Ditto.

> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> 
> How is this synchronized with the sof_count increment in irq_handle_sof()?
> 

The sof_idx field is only used for debugging purpose.
We will remove this.

> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> 
> We should check that the object type before assuming it's a vb2_buffer by
> calling vb2_request_object_is_buffer().
> 

Fix in next patch.

> > +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +		framejob->request_fd = buf->vbb.request_fd;
> 
> We shouldn't use request_fd as the key here. We already have req here, which
> is the right key to use.
> 
> That said, I can see framejob->request_fd only used for printing a debugging
> message in mtk_cam_dev_job_finish(). The request API core should already
> print something for us once a request is completed, so perhaps that isn't
> needed?
> 

Fix in next patch.

> > +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> > +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> > +		list_add_tail(&buf->list, &framejob->list_buf);
> 
> Why do we need this private list? We could just call exactly the same
> list_for_each() over the request objects.
> 

This function is re-designed.

> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> 
> We already have a list head in mtk_cam_dev_request.
> 

This function is re-designed.

> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		framejob->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> > +				      p1_dev->isp_ctx.clk_list);
> > +	if (ret)
> > +		goto clk_err;
> > +
> > +	return 0;
> > +
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> 
> clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
> this would disable the clocks second time.
> 

Fix in next patch.

> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> > +}
> 
> There is no point in having wrapper functions to just call one function
> inside. Please just call clk_bulk_*() directly.
> 

Fix in next patch.

> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check if the device isn't already runtime suspended. If it is, we
> don't have to do anything here and can just return.
> 
> 

Add pm_runtime_suspended(dev) to check.

> We also need to ensure that no new requests are queued to the hardware at
> this point. This could be done by replacing any of the kthreads with
> workqueues and making all of the workqueues freezable.
> 

Yes, we will use workqueue to send frame request.
Here is the workqueue's definition.

	p1_dev->composer_wq =
		alloc_ordered_workqueue(dev_name(p1_dev->dev),
					__WQ_LEGACY | WQ_MEM_RECLAIM |
					WQ_FREEZABLE);

> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* Disable view finder */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, the TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> 
> Are you sure this is the right handling? We need to make sure the hardware
> finishes processing the current frame and stops.
> 

We will revise this handling to make sure the ISP HW is idle before
suspend.

> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check runtime PM status here as well, because if the device was
> suspended, there is nothing to do here.
> 

Ditto.

> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev.streaming) {
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> 
> Does the hardware keep all the state, e.g. queued buffers, during suspend?
> Would the code above continue all the capture from the next buffer, as
> queued by the userspace before the suspend?
> 

Yes, we will test it.
1. SCP buffers are kept by SCP processor
2. ISP registers are still kept even if ISP clock is disable.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> 
> Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
> reference to platform_device is very useful later in the code.
> 

Fix in next patch.

> > +
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> 
> I think we want to start from 0 here?
> 

Because of single CAM support, we will revise our DTS tree to support
single CAM only. So we could remove this loop and check the CAM-B HW
information here. Here is below new function.

static int mtk_isp_probe(struct platform_device *pdev)
{
	/* List of clocks required by isp cam */
	static const char * const clk_names[] = {
		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
	};
	struct mtk_isp_p1_device *p1_dev;
	struct device *dev = &pdev->dev;
	struct resource *res;
	int irq, ret, i;

	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
	if (!p1_dev)
		return -ENOMEM;

	p1_dev->dev = dev;
	dev_set_drvdata(dev, p1_dev);

	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
	p1_dev->regs = devm_ioremap_resource(dev, res);
	if (IS_ERR(p1_dev->regs)) {
		dev_err(dev, "Failed platform resources map\n");
		return PTR_ERR(p1_dev->regs);
	}
	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);

	irq = platform_get_irq(pdev, 0);
	if (!irq) {
		dev_err(dev, "Missing IRQ resources data\n");
		return -ENODEV;
	}
	ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
			       dev_name(dev), p1_dev);
	if (ret) {
		dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
			dev->of_node->name, irq);
		return ret;
	}
	dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
	spin_lock_init(&p1_dev->spinlock_irq);

	p1_dev->num_clks = ARRAY_SIZE(clk_names);
	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
				    sizeof(*p1_dev->clks), GFP_KERNEL);
	if (!p1_dev->clks)
		return -ENOMEM;

	for (i = 0; i < p1_dev->num_clks; ++i)
		p1_dev->clks[i].id = clk_names[i];

	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
		return ret;
	}

	ret = isp_setup_scp_rproc(p1_dev, pdev);
	if (ret)
		return ret;

	pm_runtime_enable(dev);

	/* Initialize the v4l2 common part */
	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
	if (ret)
		return ret;

	return 0;
}

> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> 
> p1_dev already includes a pointer to this.
> 

Fix in next patch.

> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> > +			i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> 
> devm_ioremap_resource() returns ERR_PTR() not NULL on error.
> 

Fix in next patch.

> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* Support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* Reg & interrupts index is shifted with 1  */
> 
> The reader can already see that it's shifted from the code below. The
> comment should say _why_ it is shifted.
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			irq = platform_get_irq(pdev, i - 1);
> 
> The bindings have 3 IRQs, but we only seem to request 2 here. Is that
> expected?
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			if (irq) {
> 
> Please invert this if, so that it bails out on error. Also, this should
> check for negative errors codes, not 0.
> 

Fix in next patch.

> > +				ret = devm_request_irq(dev, irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> 
> Use dev_name().
> 

Fix in next patch.

> > +						       (void *)isp_dev);
> 
> No need to cast to void *.
> 

Fix in next patch.

> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						irq);
> > +					return ret;
> > +				}
> > +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> > +					irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> 
> We might want to move out the body of this loop to a separate function to
> keep this function shorter.
> 

Since we have already remove this loop, maybe we don't need to move out
the body into one single function.

> > +
> > +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_ctx.clk_list =
> 
> nit: "list" is the term commonly used for the list data structure. There is
> also a convention to call the length of array XXX num_XXX, so how about
> clks and num_clks?
> 

Fix in next patch.

> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_ctx.num_clks,
> > +			     sizeof(*p1_dev->isp_ctx.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_ctx.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> > +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
> 
> Shouldn't this be clk_list[i].id?
> 

Fix in next patch.

> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_ctx.num_clks,
> > +				p1_dev->isp_ctx.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> > +		goto err_init;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret)
> > +		goto err_init;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +
> > +err_init:
> > +	if (p1_dev->cam_dev.smem_dev)
> > +		device_unregister(p1_dev->cam_dev.smem_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> 
> For V4L2 drivers system and runtime PM ops would normally be completely
> different. Runtime PM ops would be called when the hardware is idle already
> or is about to become active. System PM ops would be called at system power
> state change and the hardware might be both idle or active. Please also see
> my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> 

Here is the new implementation. It should be clear to show the
difference between system and runtime PM ops. 

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
				pm_runtime_force_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
NULL)
};

> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> 
> Shouldn't this be something like mtk-cam-p1? Please make sure this
> reasonably consistent with other drivers.
> 

Fix in next patch.

> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> 
> Mediatek Camera P1 ISP driver? Please make sure this is reasonably
> consistent with other drivers (SenInf, DIP, FD).
> 
> > +MODULE_LICENSE("GPL");
> 
> GPL v2
> 

We will check this naming with other drivers & fix the version.

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6af3f569664c
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,243 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-scp.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328
> > +#define CAM_A_MAX_HEIGHT		2496
> > +#define CAM_B_MAX_WIDTH		5376
> > +#define CAM_B_MAX_HEIGHT		4032
> > +
> > +#define CAM_MIN_WIDTH			80
> > +#define CAM_MIN_HEIGHT			60
> > +
> > +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO				BIT(0)
> > +#define R_RRZO				BIT(1)
> > +#define R_AAO				BIT(3)
> > +#define R_AFO				BIT(4)
> > +#define R_LCSO				BIT(5)
> > +#define R_PDO				BIT(6)
> > +#define R_LMVO				BIT(7)
> > +#define R_FLKO				BIT(8)
> > +#define R_RSSO				BIT(9)
> > +#define R_PSO				BIT(10)
> > +
> > +#define CQ_BUFFER_COUNT		3
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> > +			"fbc1(0x%x), fbc2(0x%x)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +/* State for struct mtk_isp_p1_ctx: composer_state */
> > +enum  {
> > +	SCP_ON = 0,
> > +	SCP_OFF
> > +};
> 
> Hmm, looks like bool.
> 

This will be removed in next patch.

> > +
> > +enum {
> > +	IMG_FMT_UNKNOWN		= 0x0000,
> > +	IMG_FMT_RAW_START	= 0x2200,
> > +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> > +	IMG_FMT_BAYER10,
> > +	IMG_FMT_BAYER12,
> > +	IMG_FMT_BAYER14,
> > +	IMG_FMT_FG_BAYER8,
> > +	IMG_FMT_FG_BAYER10,
> > +	IMG_FMT_FG_BAYER12,
> > +	IMG_FMT_FG_BAYER14,
> > +};
> > +
> > +enum {
> > +	RAW_PXL_ID_B = 0,
> > +	RAW_PXL_ID_GB,
> > +	RAW_PXL_ID_GR,
> > +	RAW_PXL_ID_R
> > +};
> 
> Please use macros with explicitly assigned values for hardware/firmware ABI
> definitions.
> 

Fix in next patch.

> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> 
> Please document the structs above using kerneldoc.
> 

These structures are removed in next patch.

> > +
> > +/*
> > + * struct isp_device - the ISP device information
> > + *
> > + * @dev: Pointer to struct device
> > + * @regs: Camera ISP base register address
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @current_frame: Current frame sequence number, set when SOF
> > + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> > + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> > + * @sof_count: The accumulated SOF counter
> > + * @isp_hw_module: Identity camera A or B
> > + *
> > + */
> > +struct isp_device {
> 
> mtk_isp_device?
> 

Fix in next patch.

> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +/*
> > + * struct mtk_isp_p1_ctx - the ISP device information
> > + *
> > + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> > + * @composer_tx_thread: TX Thread for SCP data tranmission
> > + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> > + * @ipi_occupied: The total number of SCP TX data has beent sent
> > + * @scp_state: The state of SCP control
> > + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> > + * @composed_frame_id: The ack. frame sequence by SCP
> > + * @composer_deinit_thread: The de-initialized thread
> > + * @p1_enqueue_list: Queue for ISP frame buffers
> > + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> > + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> > + * @irq_data_start: Start index of irq_event_datas ring buffer
> > + * @irq_data_end: End index of irq_event_datas ring buffer
> > + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> > + * @scp_mem_pa: DMA address for SCP device
> > + * @scp_mem_iova: DMA address for ISP HW DMA devices
> > + * @frame_seq_no: Sequence number for ISP frame buffer
> > + * @isp_hw_module: Active camera HW module
> > + * @num_clks: The number of driver's clock
> > + * @clk_list: The list of clock data
> > + * @lock: Lock to protect context operations
> > + *
> > + */
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	atomic_t ipi_occupied;
> > +	atomic_t scp_state;
> > +	atomic_t composing_frame;
> > +	atomic_t composed_frame_id;
> > +	struct isp_thread composer_deinit_thread;
> > +	struct isp_queue p1_enqueue_list;
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> 
> Not documented above.
> 

Fix in next patch.

> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +	struct mutex lock; /* Protect context operations */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct mtk_cam_dev cam_dev;
> > +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> > +};
> 
> Please document in a kerneldoc comment.
> 

Fix in next patch.

> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> 
> No need to cast. And, I don't think we need a function for this, just call
> dev_get_drvdata() directly.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments .

Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-20  9:58         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-20  9:58 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi, Tomasz:

On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> >  4 files changed, 1457 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thanks for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..9e59a6bfc6b7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,126 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT			BIT(0)
> > +#define CMOS_EN_BIT			BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST			BIT(0)
> > +#define HW_PASS1_DON_ST		BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define SW_PASS1_DON_ST		BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST			BIT(4)
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +#define PSO_DONE_ST			BIT(14)
> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM		( \
> > +					VS_INT_ST |\
> > +					SOF_INT_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					SW_PASS1_DON_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM		( \
> > +					SOF_INT_ST |\
> > +					SW_PASS1_DON_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					VS_INT_ST |\
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					RRZO_ERR_ST |\
> > +					AFO_ERR_ST |\
> > +					IMGO_ERR_ST |\
> > +					AAO_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM		( \
> > +					AAO_DONE_ST |\
> > +					AFO_DONE_ST)
> 
> Could we define the values next to the addresses of registers they apply to?
> Also without the _BIT suffix and with the values prefixed with register
> names. For example:
> 
> #define REG_TG_SEN_MODE		        0x0230
> #define TG_SEN_MODE_CMOS_EN		BIT(0)
> 
> #define REG_TG_VF_CON			0x0234
> #define TG_VF_CON_VFDATA_EN		BIT(0)
> 

Fix in next patch.

> > +
> > +/* Status check */
> > +#define REG_CTL_EN			0x0004
> > +#define REG_CTL_DMA_EN			0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2			0x0018
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_VF_CON			0x0234
> > +
> > +#define REG_IMGO_BASE_ADDR		0x1020
> > +#define REG_RRZO_BASE_ADDR		0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +#define REG_AAO_ERR_STAT		0x1368
> > +#define REG_AFO_ERR_STAT		0x136c
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +#define REG_UFEO_ERR_STAT		0x1374
> > +#define REG_PDO_ERR_STAT		0x1378
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +#define REG_PDI_ERR_STAT		0x138c
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +#define REG_HW_FRAME_NUM		0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX		0x14dc
> > +#define REG_META1_VB2_INDEX		0x151c
> 
> I don't believe these registers are really for VB2 indexes.
> 

MTK P1 ISP HW supports frame header spare registers for each DMA, such
as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
frame information in these ISP registers. In this case, we save META0
VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
spare register. These implementation is designed for non-request 3A
DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as 
REG_META0_VB2_INDEX for easy understanding. Moreover, if we only need to
support request mode, we should remove this here.

cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
cmd_params.meta_frame.enabled_dma = dma_port;
cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;

> > +
> > +/* FBC */
> > +#define REG_AAO_FBC_STATUS		0x013c
> > +#define REG_AFO_FBC_STATUS		0x0134
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..c5a3babed69d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +#include <media/v4l2-event.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> 
> Please move below. Just above the platform_driver struct where it's used.
> 

Fix in next patch.

> > +
> > +/* List of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +};
> 
> Please move inside mtk_isp_probe, as a static const local variable. That
> could also let you shorten the name, to clk_names for example.
> 

Fix in next patch.

> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +					 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> 
> nit: You can just initialize the structure in the declaration.
> 

Fix in next patch.

void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
				  unsigned int frame_seq_no)
{
	struct v4l2_event event = {
		.type = V4L2_EVENT_FRAME_SYNC,
		.u.frame_sync.frame_sequence = frame_seq_no,
	};

	v4l2_event_queue(cam->subdev.devnode, &event);
}

> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +}
> > +
> > +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> > +				   unsigned int request_fd,
> > +				   unsigned int frame_seq_no,
> > +				   struct list_head *list_buf,
> > +				   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	u64    timestamp;
> 
> Too many spaces between u64 and timestamp.
> 

Fix in next patch.

> > +
> > +	if (!cam_dev->streaming)
> > +		return;
> > +
> > +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> > +		__func__, request_fd, frame_seq_no, state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	timestamp = ktime_get_ns();
> > +	list_for_each_entry_safe(buf, b0, list_buf, list) {
> 
> nit: s/b0/buf_prev/
> 

Fix in next patch.

> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = timestamp;
> > +		buf->vbb.sequence = frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> Any buffer that is not active shouldn't be on this list. If it is then it's
> a bug somewhere else in the driver. Could be possibly related to the request
> handling issues I pointed out in another comment.
> 

Fix in next patch.

> > +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct vb2_buffer *vb;
> > +
> > +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> > +		return;
> > +
> > +	spin_lock(&node->slock);
> > +	b = list_first_entry(&node->pending_list,
> > +			     struct mtk_cam_dev_buffer,
> > +			     list);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		vb = &b->vbb.vb2_buf;
> > +		if (!vb->vb2_queue->uses_requests &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vb->timestamp = ktime_get_ns();
> > +			b->vbb.sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +			list_del(&b->list);
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&node->slock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_warn(dev,
> > +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				 framejob->frame_seq_no,
> > +				 atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		} else {
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> 
> dequeue
> 

Fix in next patch.

> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> > +		    event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int addr_offset;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	if (cq_num <= frame_num) {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> > +		return cq_num;
> > +	}
> > +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> > +
> > +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(isp_dev->dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> > +		cq_num, frame_num, addr_offset);
> > +
> > +	return cq_num;
> > +}
> > +
> > +static void irq_handle_notify_event(struct isp_device *isp_dev,
> > +				    unsigned int irq_status,
> > +				    unsigned int dma_status,
> > +				    bool sof_only)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	if (irq_status & VS_INT_ST) {
> > +		/* Notify specific HW events to user space */
> > +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> > +					     isp_dev->current_frame);
> 
> Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
> definition of the V4L2_EVENT_FRAME_SYNC event at
> https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html
> 

Fix in next patch.
We will change to use SOF_INT_ST to avoid misunderstanding.

> > +		dev_dbg(dev,
> > +			"frame sync is sent:%d:%d\n",
> > +			isp_dev->sof_count,
> > +			isp_dev->current_frame);
> > +		if (sof_only)
> > +			return;
> 
> If this function can be called only to perform this block, perhaps it should
> be split into two functions?
> 
> Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
> irq_status? Is it expected that in such case the other part of the function
> is executed?
> 

Ok, we will call mtk_cam_dev_event_frame_sync function when receiving
SOF_INT_ST ISR event in the caller and remove this block.

> > +	}
> > +
> > +	if (irq_status & SW_PASS1_DON_ST) {
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask =
> > +		(irq_status & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask =
> > +		(dma_status & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> 
> I can see that all isp_deque_work() does is returning buffers to vb2. I
> don't think we need this intricate system to do that, as we could just do
> it inside the interrupt handler, in isp_irq_cam() directly.
> 

Ok, we will move all dequeue function in the ISR function and remove
this dequeue thread and related codes.

> > +
> > +	dev_dbg(dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irq_status & INT_ST_MASK_CAM),
> > +		(dma_status & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cam_idx, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned int aao_fbc, afo_fbc;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev.streaming)
> > +		return IRQ_HANDLED;
> 
> This shouldn't be needed. The driver needs to unmask the interrupts in
> hardware registers when it starts streaming and mask them when it stops.
> Note that I mean the P1 hardware registers, not disable_irq(), which
> shouldn't be used.
> 

Fix in next patch.

> > +
> > +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> 
> Hmm, reading vb2 buffer index from hardware registers? Was this hardware
> designed exclusively for V4L2? ;)
> 
> Jokes aside, how does the hardware know V4L2 buffer indexes?
> 

This is explained in the above.

> > +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> > +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> > +		return IRQ_HANDLED;
> 
> Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
> get an interrupt without any unmasked hardware IRQs active in the status,
> that's an error somewhere and we should return IRQ_NONE.
> 

Ok, we will check the IRQ EN register firstly and check any unmasked
IRQs for IRQ_NONE case.

> > +
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +
> > +	/* Sof, done order check */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue frame */
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > +		isp_dev->current_frame = hw_frame_num;
> 
> What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> event?
> 

This is a another spare register for frame sequence number usage.
It comes from struct p1_frame_param:frame_seq_no which is sent by
SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
Is it a better understanding?

Below is our frame request handling in current design.

1. Buffer preparation
- Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
other meta histogram buffers (LCSO/LMVO) into one request.
- Accumulated one unique frame sequence number to each request and send
this request to the SCP composer to compose CQ (Command queue) buffer
via SCP_ISP_FRAME IPI command.
- CQ buffer is frame registers set. If ISP registers should be updated
per frame, these registers are configured in the CQ buffer, such as
frame sequence number, DMA addresses and tuning ISP registers.
- One frame request will be composed into one CQ buffer.Once CQ buffer
is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
its corresponding frame sequence number. Based on this, kernel driver
knows which request is ready to be en-queued and save this with
p1_dev->isp_ctx.composed_frame_id.
- The maximum number of CQ buffers in SCP is 3.

2. Buffer en-queue flow
- In order to configure correct CQ buffer setting before next SQF event,
it is depended on by MTK ISP P1 HW CQ mechanism.
- The basic concept of CQ mechanism is loaded ISP CQ buffer settings
when HW_PASS1_DON_ST is received which means DMA output is done.
- Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
address is used. Otherwise, it will loaded one dummy CQ buffer to
bypass.
- So we will check available CQ buffers by comparing composed frame
sequence number & dequeued frame sequence from ISP HW in SOF event.
- If there are available CQ buffers, update the CQ base address to the
next CQ buffer address based on current de-enqueue frame sequence
number. So MTK ISP P1 HW will load this CQ buffer into HW when
HW_PASS1_DON_ST is triggered which is before the next SOF.
- So in next SOF event, ISP HW starts to output DMA buffers with this
request until request is done.
- But, for the first request, it is loaded into HW manually when
streaming is on for better performance.

3. Buffer de-queue flow
- We will use frame sequence number to decide which request is ready to
de-queue.
- We will save some important register setting from ISP HW when SOF is
received. This is because the ISP HW starts to output the data with the
corresponding settings, especially frame sequence number setting.
- When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
done. So we could call isp_deque_request_frame with frame sequence
number to de-queue frame to VB2
- For AAO/AFO buffers, it has similar design concept. Sometimes, if only
AAO/AFO non-request buffers are en-queued without request buffers at the
same time, there will be no SW P1 done event for AAO/AFO DMA done.
Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
- Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
we may try to send another request to SCP for composing.

Hopefully, my explanation is helpful for better understanding our
implementation. If you still have any questions, please let me know. 

> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irq_status & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > +	}
> 
> The if and else blocks do almost the same things just in different order. Is
> it really expected?
> 

If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
the correct sequence should be handle HW_PASS1_DON_ST firstly to check
any de-queued frame and update the next frame setting later.
Normally, this is a corner case or system performance issue.

Btw, we will revise the above implementation as below.


if (irq_status & SOF_INT_ST)
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
					     dequeue_frame_seq_no);

/* Sof, done order check */
if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
	dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

/* Notify IRQ event and de-enqueue frame */
irq_handle_notify_event(p1_dev, irq_status, dma_status);

/* Update frame settings & CQ address for frame en-queue */
enqueue_frame_seq_no = 0;
if (irq_status & SOF_INT_ST)
	enqueue_frame_seq_no = irq_handle_sof(p1_dev,
					      dequeue_frame_seq_no,
					      meta0_vb2_index,
					      meta1_vb2_index); 

> > +
> > +	if (irq_status & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x\n",
> > +			irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irq_status & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> 
> Please just put that string here, otherwise the reader would have no idea
> what message is being printed here.
> 

Fix in next patch.

> > +			'A' + cam_idx,
> > +			isp_dev->sof_count,
> > +			irq_status,
> > +			dma_status,
> > +			hw_frame_num,
> > +			cq_num,
> > +			aao_fbc,
> > +			afo_fbc);
> 
> nit: No need to put each argument in its own line.
> 

Fix in next patch.

> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> 
> This look-up should be done once in probe(). Only the rproc_boot() should
> happen dynamically.
> 

Fix in next patch.

> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +	mutex_init(&isp_ctx->lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +
> > +	return 0;
> 
> Everything here looks like something that should be done once in probe. I
> also don't see a point of having a separate mtk_isp_p1_ctx struct for the
> data above. It could be just located in p1_dev, at least for now.
> 
> If we end up implementing support for multiple contexts, we could isolate
> per-context data then, once we know what's really per-context. For now,
> let's keep it simple.
> 

Ok, we will remove isp_ctx structure and move some fields into p1_dev or
cam_dev,

> > +}
> > +
> > +static int isp_uninit_context(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	if (isp_ctx->isp_deque_thread.thread) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	mutex_destroy(&isp_ctx->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int enabled_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enabled_dma_ports = 0;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		if (cam_dev->vdev_nodes[i].enabled)
> > +			enabled_dma_ports |=
> > +				cam_dev->vdev_nodes[i].desc.dma_port;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> > +
> > +	return enabled_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return RAW_PXL_ID_R;
> > +	default:
> 
> Could we fail here instead?
> 

Ok, we will return invalid value here, like MTK_CAM_RAW_PXL_ID_UNKNOWN.
We will also check this error in the caller.

> > +		return RAW_PXL_ID_B;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return IMG_FMT_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> 
> Could we fail here instead, so that we don't mask some potential errors?
> 

Ok, we will return MTK_CAM_IMG_FMT_UNKNOWN and check this error in the
caller.

> > +		return 10;
> > +	}
> > +}
> > +
> > +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> > +			   const struct v4l2_format *in_fmt,
> > +			   const struct v4l2_subdev_format *sd_format)
> > +{
> > +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> > +
> > +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> 
> Please group operations on the same field together, i.e. remove the blank
> line above size.stride and add one blank line above size.w.
> 

Fix in next patch.

> > +
> > +	out_fmt->crop.left = 0x0;
> > +	out_fmt->crop.top = 0x0;
> > +
> 
> Remove the blank line.
> 

Fix in next patch.

> > +	out_fmt->crop.width = sd_format->format.width;
> > +	out_fmt->crop.height = sd_format->format.height;
> > +
> > +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> > +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> > +		  "img out:%d:%d in:%d:%d",
> > +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> > +		  out_fmt->crop.width, out_fmt->crop.height);
> 
> We should check this earlier and fail the streaming start if there is a
> mismatch between sensor and video node configuration.
> 

Fix in next patch.

> > +
> > +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> > +		out_fmt->pixel_byte,
> > +		out_fmt->img_fmt);
> > +	dev_dbg(dev,
> > +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = isp_init_context(p1_dev);
> > +	if (ret)
> > +		return ret;
> 
> The above function doesn't really seem to be related to power management.
> Should it be called from subdev stream on?
> 

We will rename this function to mtk_isp_hw_init.
But, it will be called when the first video node is streamed on.
This is because we need to initialize the HW firstly for sub-device
stream-on performance.  We need to send some IPI commands, such as
ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
timing.

> > +
> > +	ret = isp_composer_init(dev);
> > +	if (ret)
> > +		goto composer_err;
> 
> This also doesn't seem to be related to power management.
> 

Ok, we will rename to mtk_isp_hw_init to avoid misunderstanding.

> > +
> > +	pm_runtime_get_sync(dev);
> > +
> > +	/* ISP HW INIT */
> > +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> 
> There is some amount of code handling various values of isp_hw_module in
> this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
> code that can't be tested. Please either add support for all the indexes in
> the driver or simplify all the code to just handle CAM_B.
> 

Ok, we will simplify driver code to support single CAM only.
We will remove isp_hw_module usage in the source code.

> > +	/* Use pure RAW as default HW path */
> > +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > +
> > +	isp_composer_hw_init(dev);
> > +	/* Check enabled DMAs which is configured by media setup */
> > +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> 
> Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> necessary?
> 

Yes, it is necessary for non-request buffers design for Camera 3A video
nodes. For 3A video nodes, we just want to know which 3A video nodes are
enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
format from user space. So we just pass the enabled DMA information from
kernel to SCP only. With 3A enabled DMA information, we could configure
related 3A registers in SCP.

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +
> > +composer_err:
> > +	isp_uninit_context(dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_power_release(struct device *dev)
> > +{
> > +	isp_composer_hw_deinit(dev);
> > +	isp_uninit_context(dev);
> 
> These two don't seem to be related to power either.
> 
> Instead, I don't see anything that could undo the rproc_boot() operation
> here.
> 

We will rename this function to mtk_isp_hw_release.
We will also add rproc_shutdown function call here.

int mtk_isp_hw_release(struct mtk_cam_dev *cam)
{
	struct device *dev = cam->dev;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	isp_composer_hw_deinit(p1_dev);
	pm_runtime_put_sync_autosuspend(dev);
	rproc_shutdown(p1_dev->rproc_handle);

	dev_dbg(dev, "%s done\n", __func__);

	return 0;
}

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	unsigned int enabled_dma_ports;
> > +	struct v4l2_format *img_fmt;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> > +
> > +	/* Sensor config */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +
> 
> Unnecessary blank line.
> 

Fix in next patch.

> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix in next patch.

> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_fmt.format.width, sd_fmt.format.height,
> > +		sd_fmt.format.code, sd_fmt.format.field,
> > +		sd_fmt.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* Fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = 0x1;
> > +	/* Support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> 
> Why hexadecimal numerals? Also, what's the meaning of these values? For
> anything boolean, you could just use true and false as a substitute of 0 and
> 1. For anything that has more values, please define the values using macros.
> 

Fix in next patch.
1. Remove unnecessary hexadecimal number usage
2. Use boolean value if possible
3. Assign non-empty field values.

Below is the fixed version.

	config_param.cfg_in_param.continuous = true;
	/* Fix to one pixel mode in default */
	config_param.cfg_in_param.pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
	config_param.cfg_in_param.raw_pixel_id =
		get_sensor_pixel_id(sd_fmt.format.code);
	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
	if (config_param.cfg_in_param.img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
	    config_param.cfg_in_param.raw_pixel_id ==
MTK_CAM_RAW_PXL_ID_UNKNOWN) {
		dev_err(dev, "Unknown cfg img fmt or unknown raw pixel id\n");
		return -EINVAL;
	}

> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_fmt.format.code);
> > +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
> 
> Move the other crop settings from above to here.
> 

Ditto.

> > +
> > +	config_param.cfg_main_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
> 
> No need for the == R_IMGO part.
> 

Since R_IMGO is mandatory video node, we will remove this checking.

> > +		config_param.cfg_main_param.bypass = 0;
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	config_param.cfg_resize_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
> 
> Ditto.
> 

Fix in next patch as below:

if (enabled_dma_ports & R_RRZO) {
	ret = config_img_fmt(cam,
			     &config_param.cfg_resize_param.output,
			     img_fmt, &sd_fmt);
	if (ret)
		return ret;
} else {
	config_param.cfg_resize_param.bypass = true;
}

> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
> 
> Should image DMAs be masked out of this bitmap?
> 

We will replace struct cfg_meta_param with enabled_dmas integer.
So we can pass all enabled DMA masks to SCP. 

> > +
> > +	isp_composer_hw_config(dev, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> > +		     struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> > +}
> > +
> > +void mtk_isp_req_flush_buffers(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_queue_job *job, *j0;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> > +
> > +	if (!atomic_read(&p1_list->queue_cnt))
> > +		return;
> 
> Do we need this explicit check? The code below wouldn't do anything if there
> isn't anything in the list. IMHO we could even remove queue_cnt completely.
> 

Since we have redesign request en-queue mechanism, this function will be
removed.

> > +
> > +	spin_lock(&p1_list->lock);
> > +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
> 
> nit: s/j0/job_prev/
> 

Will apply this naming rule in next patch.

> > +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
> 
> nit: s/b0/buf_pref/
> 

Ditto.

> Also, we should be able to replace this with iterating over the generic list
> of request objects, rather than this internal list.
> 
> > +			list_del(&buf->list);
> > +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> It shouldn't be possible to happen. If you see this check failing, that
> means a problem somewhere else in the driver.
> 

Fix in next patch.

> > +				vb2_buffer_done(&buf->vbb.vb2_buf,
> > +						VB2_BUF_STATE_ERROR);
> > +		}
> > +		list_del(&job->list_entry);
> > +		atomic_dec(&p1_list->queue_cnt);
> > +		kfree(job);
> > +	}
> > +	spin_unlock(&p1_list->lock);
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

We got your point.
Below is new function declaration.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct media_request_object *obj, *obj_safe;
> > +	struct vb2_buffer *vb;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> 
> This allocation shouldn't be needed. The structure should be already a part
> of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> even say that the contents of this struct should be just moved to that one
> to avoid overabstracting.
> 

For this function, we will apply the new design from P2 driver's
comment. Here is the new implementation.

struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

> > +	memset(framejob, 0, sizeof(*framejob));
> 
> Putting the above comment aside, kzalloc() already zeroes the memory.
> 

Ditto.

> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> 
> How is this synchronized with the sof_count increment in irq_handle_sof()?
> 

The sof_idx field is only used for debugging purpose.
We will remove this.

> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> 
> We should check that the object type before assuming it's a vb2_buffer by
> calling vb2_request_object_is_buffer().
> 

Fix in next patch.

> > +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +		framejob->request_fd = buf->vbb.request_fd;
> 
> We shouldn't use request_fd as the key here. We already have req here, which
> is the right key to use.
> 
> That said, I can see framejob->request_fd only used for printing a debugging
> message in mtk_cam_dev_job_finish(). The request API core should already
> print something for us once a request is completed, so perhaps that isn't
> needed?
> 

Fix in next patch.

> > +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> > +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> > +		list_add_tail(&buf->list, &framejob->list_buf);
> 
> Why do we need this private list? We could just call exactly the same
> list_for_each() over the request objects.
> 

This function is re-designed.

> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> 
> We already have a list head in mtk_cam_dev_request.
> 

This function is re-designed.

> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		framejob->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> > +				      p1_dev->isp_ctx.clk_list);
> > +	if (ret)
> > +		goto clk_err;
> > +
> > +	return 0;
> > +
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> 
> clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
> this would disable the clocks second time.
> 

Fix in next patch.

> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> > +}
> 
> There is no point in having wrapper functions to just call one function
> inside. Please just call clk_bulk_*() directly.
> 

Fix in next patch.

> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check if the device isn't already runtime suspended. If it is, we
> don't have to do anything here and can just return.
> 
> 

Add pm_runtime_suspended(dev) to check.

> We also need to ensure that no new requests are queued to the hardware at
> this point. This could be done by replacing any of the kthreads with
> workqueues and making all of the workqueues freezable.
> 

Yes, we will use workqueue to send frame request.
Here is the workqueue's definition.

	p1_dev->composer_wq =
		alloc_ordered_workqueue(dev_name(p1_dev->dev),
					__WQ_LEGACY | WQ_MEM_RECLAIM |
					WQ_FREEZABLE);

> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* Disable view finder */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, the TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> 
> Are you sure this is the right handling? We need to make sure the hardware
> finishes processing the current frame and stops.
> 

We will revise this handling to make sure the ISP HW is idle before
suspend.

> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check runtime PM status here as well, because if the device was
> suspended, there is nothing to do here.
> 

Ditto.

> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev.streaming) {
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> 
> Does the hardware keep all the state, e.g. queued buffers, during suspend?
> Would the code above continue all the capture from the next buffer, as
> queued by the userspace before the suspend?
> 

Yes, we will test it.
1. SCP buffers are kept by SCP processor
2. ISP registers are still kept even if ISP clock is disable.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> 
> Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
> reference to platform_device is very useful later in the code.
> 

Fix in next patch.

> > +
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> 
> I think we want to start from 0 here?
> 

Because of single CAM support, we will revise our DTS tree to support
single CAM only. So we could remove this loop and check the CAM-B HW
information here. Here is below new function.

static int mtk_isp_probe(struct platform_device *pdev)
{
	/* List of clocks required by isp cam */
	static const char * const clk_names[] = {
		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
	};
	struct mtk_isp_p1_device *p1_dev;
	struct device *dev = &pdev->dev;
	struct resource *res;
	int irq, ret, i;

	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
	if (!p1_dev)
		return -ENOMEM;

	p1_dev->dev = dev;
	dev_set_drvdata(dev, p1_dev);

	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
	p1_dev->regs = devm_ioremap_resource(dev, res);
	if (IS_ERR(p1_dev->regs)) {
		dev_err(dev, "Failed platform resources map\n");
		return PTR_ERR(p1_dev->regs);
	}
	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);

	irq = platform_get_irq(pdev, 0);
	if (!irq) {
		dev_err(dev, "Missing IRQ resources data\n");
		return -ENODEV;
	}
	ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
			       dev_name(dev), p1_dev);
	if (ret) {
		dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
			dev->of_node->name, irq);
		return ret;
	}
	dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
	spin_lock_init(&p1_dev->spinlock_irq);

	p1_dev->num_clks = ARRAY_SIZE(clk_names);
	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
				    sizeof(*p1_dev->clks), GFP_KERNEL);
	if (!p1_dev->clks)
		return -ENOMEM;

	for (i = 0; i < p1_dev->num_clks; ++i)
		p1_dev->clks[i].id = clk_names[i];

	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
		return ret;
	}

	ret = isp_setup_scp_rproc(p1_dev, pdev);
	if (ret)
		return ret;

	pm_runtime_enable(dev);

	/* Initialize the v4l2 common part */
	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
	if (ret)
		return ret;

	return 0;
}

> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> 
> p1_dev already includes a pointer to this.
> 

Fix in next patch.

> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> > +			i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> 
> devm_ioremap_resource() returns ERR_PTR() not NULL on error.
> 

Fix in next patch.

> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* Support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* Reg & interrupts index is shifted with 1  */
> 
> The reader can already see that it's shifted from the code below. The
> comment should say _why_ it is shifted.
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			irq = platform_get_irq(pdev, i - 1);
> 
> The bindings have 3 IRQs, but we only seem to request 2 here. Is that
> expected?
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			if (irq) {
> 
> Please invert this if, so that it bails out on error. Also, this should
> check for negative errors codes, not 0.
> 

Fix in next patch.

> > +				ret = devm_request_irq(dev, irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> 
> Use dev_name().
> 

Fix in next patch.

> > +						       (void *)isp_dev);
> 
> No need to cast to void *.
> 

Fix in next patch.

> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						irq);
> > +					return ret;
> > +				}
> > +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> > +					irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> 
> We might want to move out the body of this loop to a separate function to
> keep this function shorter.
> 

Since we have already remove this loop, maybe we don't need to move out
the body into one single function.

> > +
> > +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_ctx.clk_list =
> 
> nit: "list" is the term commonly used for the list data structure. There is
> also a convention to call the length of array XXX num_XXX, so how about
> clks and num_clks?
> 

Fix in next patch.

> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_ctx.num_clks,
> > +			     sizeof(*p1_dev->isp_ctx.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_ctx.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> > +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
> 
> Shouldn't this be clk_list[i].id?
> 

Fix in next patch.

> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_ctx.num_clks,
> > +				p1_dev->isp_ctx.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> > +		goto err_init;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret)
> > +		goto err_init;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +
> > +err_init:
> > +	if (p1_dev->cam_dev.smem_dev)
> > +		device_unregister(p1_dev->cam_dev.smem_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> 
> For V4L2 drivers system and runtime PM ops would normally be completely
> different. Runtime PM ops would be called when the hardware is idle already
> or is about to become active. System PM ops would be called at system power
> state change and the hardware might be both idle or active. Please also see
> my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> 

Here is the new implementation. It should be clear to show the
difference between system and runtime PM ops. 

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
				pm_runtime_force_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
NULL)
};

> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> 
> Shouldn't this be something like mtk-cam-p1? Please make sure this
> reasonably consistent with other drivers.
> 

Fix in next patch.

> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> 
> Mediatek Camera P1 ISP driver? Please make sure this is reasonably
> consistent with other drivers (SenInf, DIP, FD).
> 
> > +MODULE_LICENSE("GPL");
> 
> GPL v2
> 

We will check this naming with other drivers & fix the version.

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6af3f569664c
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,243 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-scp.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328
> > +#define CAM_A_MAX_HEIGHT		2496
> > +#define CAM_B_MAX_WIDTH		5376
> > +#define CAM_B_MAX_HEIGHT		4032
> > +
> > +#define CAM_MIN_WIDTH			80
> > +#define CAM_MIN_HEIGHT			60
> > +
> > +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO				BIT(0)
> > +#define R_RRZO				BIT(1)
> > +#define R_AAO				BIT(3)
> > +#define R_AFO				BIT(4)
> > +#define R_LCSO				BIT(5)
> > +#define R_PDO				BIT(6)
> > +#define R_LMVO				BIT(7)
> > +#define R_FLKO				BIT(8)
> > +#define R_RSSO				BIT(9)
> > +#define R_PSO				BIT(10)
> > +
> > +#define CQ_BUFFER_COUNT		3
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> > +			"fbc1(0x%x), fbc2(0x%x)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +/* State for struct mtk_isp_p1_ctx: composer_state */
> > +enum  {
> > +	SCP_ON = 0,
> > +	SCP_OFF
> > +};
> 
> Hmm, looks like bool.
> 

This will be removed in next patch.

> > +
> > +enum {
> > +	IMG_FMT_UNKNOWN		= 0x0000,
> > +	IMG_FMT_RAW_START	= 0x2200,
> > +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> > +	IMG_FMT_BAYER10,
> > +	IMG_FMT_BAYER12,
> > +	IMG_FMT_BAYER14,
> > +	IMG_FMT_FG_BAYER8,
> > +	IMG_FMT_FG_BAYER10,
> > +	IMG_FMT_FG_BAYER12,
> > +	IMG_FMT_FG_BAYER14,
> > +};
> > +
> > +enum {
> > +	RAW_PXL_ID_B = 0,
> > +	RAW_PXL_ID_GB,
> > +	RAW_PXL_ID_GR,
> > +	RAW_PXL_ID_R
> > +};
> 
> Please use macros with explicitly assigned values for hardware/firmware ABI
> definitions.
> 

Fix in next patch.

> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> 
> Please document the structs above using kerneldoc.
> 

These structures are removed in next patch.

> > +
> > +/*
> > + * struct isp_device - the ISP device information
> > + *
> > + * @dev: Pointer to struct device
> > + * @regs: Camera ISP base register address
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @current_frame: Current frame sequence number, set when SOF
> > + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> > + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> > + * @sof_count: The accumulated SOF counter
> > + * @isp_hw_module: Identity camera A or B
> > + *
> > + */
> > +struct isp_device {
> 
> mtk_isp_device?
> 

Fix in next patch.

> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +/*
> > + * struct mtk_isp_p1_ctx - the ISP device information
> > + *
> > + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> > + * @composer_tx_thread: TX Thread for SCP data tranmission
> > + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> > + * @ipi_occupied: The total number of SCP TX data has beent sent
> > + * @scp_state: The state of SCP control
> > + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> > + * @composed_frame_id: The ack. frame sequence by SCP
> > + * @composer_deinit_thread: The de-initialized thread
> > + * @p1_enqueue_list: Queue for ISP frame buffers
> > + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> > + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> > + * @irq_data_start: Start index of irq_event_datas ring buffer
> > + * @irq_data_end: End index of irq_event_datas ring buffer
> > + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> > + * @scp_mem_pa: DMA address for SCP device
> > + * @scp_mem_iova: DMA address for ISP HW DMA devices
> > + * @frame_seq_no: Sequence number for ISP frame buffer
> > + * @isp_hw_module: Active camera HW module
> > + * @num_clks: The number of driver's clock
> > + * @clk_list: The list of clock data
> > + * @lock: Lock to protect context operations
> > + *
> > + */
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	atomic_t ipi_occupied;
> > +	atomic_t scp_state;
> > +	atomic_t composing_frame;
> > +	atomic_t composed_frame_id;
> > +	struct isp_thread composer_deinit_thread;
> > +	struct isp_queue p1_enqueue_list;
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> 
> Not documented above.
> 

Fix in next patch.

> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +	struct mutex lock; /* Protect context operations */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct mtk_cam_dev cam_dev;
> > +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> > +};
> 
> Please document in a kerneldoc comment.
> 

Fix in next patch.

> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> 
> No need to cast. And, I don't think we need a function for this, just call
> dev_get_drvdata() directly.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments .

Best regards,


Jungo




^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-20  9:58         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-20  9:58 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi, Tomasz:

On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other functions.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> >  4 files changed, 1457 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thanks for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..9e59a6bfc6b7
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,126 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _CAM_REGS_H
> > +#define _CAM_REGS_H
> > +
> > +/* TG Bit Mask */
> > +#define VFDATA_EN_BIT			BIT(0)
> > +#define CMOS_EN_BIT			BIT(0)
> > +
> > +/* normal signal bit */
> > +#define VS_INT_ST			BIT(0)
> > +#define HW_PASS1_DON_ST		BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define SW_PASS1_DON_ST		BIT(30)
> > +
> > +/* err status bit */
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +
> > +/* CAM DMA done status */
> > +#define FLKO_DONE_ST			BIT(4)
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +#define PSO_DONE_ST			BIT(14)
> > +
> > +/* IRQ signal mask */
> > +#define INT_ST_MASK_CAM		( \
> > +					VS_INT_ST |\
> > +					SOF_INT_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					SW_PASS1_DON_ST)
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* IRQ Signal Log Mask */
> > +#define INT_ST_LOG_MASK_CAM		( \
> > +					SOF_INT_ST |\
> > +					SW_PASS1_DON_ST |\
> > +					HW_PASS1_DON_ST |\
> > +					VS_INT_ST |\
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					RRZO_ERR_ST |\
> > +					AFO_ERR_ST |\
> > +					IMGO_ERR_ST |\
> > +					AAO_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +/* DMA Event Notification Mask */
> > +#define DMA_ST_MASK_CAM		( \
> > +					AAO_DONE_ST |\
> > +					AFO_DONE_ST)
> 
> Could we define the values next to the addresses of registers they apply to?
> Also without the _BIT suffix and with the values prefixed with register
> names. For example:
> 
> #define REG_TG_SEN_MODE		        0x0230
> #define TG_SEN_MODE_CMOS_EN		BIT(0)
> 
> #define REG_TG_VF_CON			0x0234
> #define TG_VF_CON_VFDATA_EN		BIT(0)
> 

Fix in next patch.

> > +
> > +/* Status check */
> > +#define REG_CTL_EN			0x0004
> > +#define REG_CTL_DMA_EN			0x0008
> > +#define REG_CTL_FMT_SEL		0x0010
> > +#define REG_CTL_EN2			0x0018
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +
> > +#define REG_TG_SEN_MODE		0x0230
> > +#define REG_TG_VF_CON			0x0234
> > +
> > +#define REG_IMGO_BASE_ADDR		0x1020
> > +#define REG_RRZO_BASE_ADDR		0x1050
> > +
> > +/* Error status log */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +#define REG_AAO_ERR_STAT		0x1368
> > +#define REG_AFO_ERR_STAT		0x136c
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +#define REG_UFEO_ERR_STAT		0x1374
> > +#define REG_PDO_ERR_STAT		0x1378
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +#define REG_PDI_ERR_STAT		0x138c
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* ISP command */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +#define REG_HW_FRAME_NUM		0x13b8
> > +
> > +/* META */
> > +#define REG_META0_VB2_INDEX		0x14dc
> > +#define REG_META1_VB2_INDEX		0x151c
> 
> I don't believe these registers are really for VB2 indexes.
> 

MTK P1 ISP HW supports frame header spare registers for each DMA, such
as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
frame information in these ISP registers. In this case, we save META0
VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
spare register. These implementation is designed for non-request 3A
DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as 
REG_META0_VB2_INDEX for easy understanding. Moreover, if we only need to
support request mode, we should remove this here.

cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
cmd_params.meta_frame.enabled_dma = dma_port;
cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;

> > +
> > +/* FBC */
> > +#define REG_AAO_FBC_STATUS		0x013c
> > +#define REG_AFO_FBC_STATUS		0x0134
> > +
> > +#endif	/* _CAM_REGS_H */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..c5a3babed69d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,1087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/cdev.h>
> > +#include <linux/compat.h>
> > +#include <linux/fs.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/ktime.h>
> > +#include <linux/module.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched/clock.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +#include <media/v4l2-event.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-regs.h"
> > +#include "mtk_cam-smem.h"
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> 
> Please move below. Just above the platform_driver struct where it's used.
> 

Fix in next patch.

> > +
> > +/* List of clocks required by isp cam */
> > +static const char * const mtk_isp_clks[] = {
> > +	"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +};
> 
> Please move inside mtk_isp_probe, as a static const local variable. That
> could also let you shorten the name, to clk_names for example.
> 

Fix in next patch.

> > +
> > +static void isp_dump_dma_status(struct isp_device *isp_dev)
> > +{
> > +	dev_err(isp_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(isp_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(isp_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(isp_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(isp_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(isp_dev->regs + REG_LSCI_ERR_STAT));
> > +}
> > +
> > +static void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +					 __u32 frame_seq_no)
> > +{
> > +	struct v4l2_event event;
> > +
> > +	memset(&event, 0, sizeof(event));
> > +	event.type = V4L2_EVENT_FRAME_SYNC;
> > +	event.u.frame_sync.frame_sequence = frame_seq_no;
> 
> nit: You can just initialize the structure in the declaration.
> 

Fix in next patch.

void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
				  unsigned int frame_seq_no)
{
	struct v4l2_event event = {
		.type = V4L2_EVENT_FRAME_SYNC,
		.u.frame_sync.frame_sequence = frame_seq_no,
	};

	v4l2_event_queue(cam->subdev.devnode, &event);
}

> > +	v4l2_event_queue(cam_dev->subdev.devnode, &event);
> > +}
> > +
> > +static void mtk_cam_dev_job_finish(struct mtk_isp_p1_ctx *isp_ctx,
> > +				   unsigned int request_fd,
> > +				   unsigned int frame_seq_no,
> > +				   struct list_head *list_buf,
> > +				   enum vb2_buffer_state state)
> > +{
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	u64    timestamp;
> 
> Too many spaces between u64 and timestamp.
> 

Fix in next patch.

> > +
> > +	if (!cam_dev->streaming)
> > +		return;
> > +
> > +	dev_dbg(&p1_dev->pdev->dev, "%s request fd:%d frame_seq:%d state:%d\n",
> > +		__func__, request_fd, frame_seq_no, state);
> > +
> > +	/*
> > +	 * Set the buffer's VB2 status so that the user can dequeue
> > +	 * the buffer.
> > +	 */
> > +	timestamp = ktime_get_ns();
> > +	list_for_each_entry_safe(buf, b0, list_buf, list) {
> 
> nit: s/b0/buf_prev/
> 

Fix in next patch.

> > +		list_del(&buf->list);
> > +		buf->vbb.vb2_buf.timestamp = timestamp;
> > +		buf->vbb.sequence = frame_seq_no;
> > +		if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> Any buffer that is not active shouldn't be on this list. If it is then it's
> a bug somewhere else in the driver. Could be possibly related to the request
> handling issues I pointed out in another comment.
> 

Fix in next patch.

> > +			vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +static void isp_deque_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +			    unsigned int node_id, int vb2_index,
> > +			    int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_cam_video_device *node = &cam_dev->vdev_nodes[node_id];
> > +	struct mtk_cam_dev_buffer *b, *b0;
> > +	struct vb2_buffer *vb;
> > +
> > +	if (!cam_dev->vdev_nodes[node_id].enabled || !cam_dev->streaming)
> > +		return;
> > +
> > +	spin_lock(&node->slock);
> > +	b = list_first_entry(&node->pending_list,
> > +			     struct mtk_cam_dev_buffer,
> > +			     list);
> > +	list_for_each_entry_safe(b, b0, &node->pending_list, list) {
> > +		vb = &b->vbb.vb2_buf;
> > +		if (!vb->vb2_queue->uses_requests &&
> > +		    vb->index == vb2_index &&
> > +		    vb->state == VB2_BUF_STATE_ACTIVE) {
> > +			dev_dbg(dev, "%s:%d:%d", __func__, node_id, vb2_index);
> > +			vb->timestamp = ktime_get_ns();
> > +			b->vbb.sequence = frame_seq_no;
> > +			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> > +			list_del(&b->list);
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&node->slock);
> > +}
> > +
> > +static void isp_deque_request_frame(struct isp_p1_device *p1_dev,
> 
> dequeue
> 

Fix in next patch.

> > +				    int frame_seq_no)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_job *framejob, *tmp;
> > +	struct isp_queue *p1_enqueue_list = &isp_ctx->p1_enqueue_list;
> > +
> > +	/* Match dequeue work and enqueue frame */
> > +	spin_lock(&p1_enqueue_list->lock);
> > +	list_for_each_entry_safe(framejob, tmp, &p1_enqueue_list->queue,
> > +				 list_entry) {
> > +		dev_dbg(dev,
> > +			"%s frame_seq_no:%d, target frame_seq_no:%d\n",
> > +			__func__,
> > +			framejob->frame_seq_no, frame_seq_no);
> > +		/* Match by the en-queued request number */
> > +		if (framejob->frame_seq_no == frame_seq_no) {
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_DONE);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_dbg(dev,
> > +				"frame_seq_no:%d is done, queue_cnt:%d\n",
> > +				framejob->frame_seq_no,
> > +				atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only when frame ready */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +			break;
> > +		} else if (framejob->frame_seq_no < frame_seq_no) {
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_finish(isp_ctx,
> > +					       framejob->request_fd,
> > +					       framejob->frame_seq_no,
> > +					       &framejob->list_buf,
> > +					       VB2_BUF_STATE_ERROR);
> > +			atomic_dec(&p1_enqueue_list->queue_cnt);
> > +			dev_warn(dev,
> > +				 "frame_seq_no:%d drop, queue_cnt:%d\n",
> > +				 framejob->frame_seq_no,
> > +				 atomic_read(&p1_enqueue_list->queue_cnt));
> > +
> > +			/* Remove only drop frame */
> > +			list_del(&framejob->list_entry);
> > +			kfree(framejob);
> > +		} else {
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&p1_enqueue_list->lock);
> > +}
> > +
> > +static int isp_deque_work(void *data)
> 
> dequeue
> 

Fix in next patch.

> > +{
> > +	struct isp_p1_device *p1_dev = (struct isp_p1_device *)data;
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_cam_dev_stat_event_data event_data;
> > +	atomic_t *irq_data_end = &isp_ctx->irq_data_end;
> > +	atomic_t *irq_data_start = &isp_ctx->irq_data_start;
> > +	unsigned long flags;
> > +	int ret, i;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible(isp_ctx->isp_deque_thread.wq,
> > +					       (atomic_read(irq_data_end) !=
> > +					       atomic_read(irq_data_start)) ||
> > +					       kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +		i = atomic_read(&isp_ctx->irq_data_start);
> > +		memcpy(&event_data, &isp_ctx->irq_event_datas[i],
> > +		       sizeof(event_data));
> > +		atomic_set(&isp_ctx->irq_data_start, ++i & 0x3);
> > +		spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +		if (event_data.irq_status_mask & HW_PASS1_DON_ST &&
> > +		    event_data.dma_status_mask & AAO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.dma_status_mask & AFO_DONE_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +		}
> > +		if (event_data.irq_status_mask & SW_PASS1_DON_ST) {
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_0,
> > +					event_data.meta0_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_frame(p1_dev,
> > +					MTK_CAM_P1_META_OUT_1,
> > +					event_data.meta1_vb2_index,
> > +					event_data.frame_seq_no);
> > +			isp_deque_request_frame(p1_dev,
> > +						event_data.frame_seq_no);
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int irq_handle_sof(struct isp_device *isp_dev,
> > +			  dma_addr_t base_addr,
> > +			  unsigned int frame_num)
> > +{
> > +	unsigned int addr_offset;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	int cq_num = atomic_read(&p1_dev->isp_ctx.composed_frame_id);
> > +
> > +	isp_dev->sof_count += 1;
> > +
> > +	if (cq_num <= frame_num) {
> > +		dev_dbg(isp_dev->dev,
> > +			"SOF_INT_ST, wait next, cq_num:%d, frame_num:%d",
> > +			cq_num, frame_num);
> > +		atomic_set(&p1_dev->isp_ctx.composing_frame, 0);
> > +		return cq_num;
> > +	}
> > +	atomic_set(&p1_dev->isp_ctx.composing_frame, cq_num - frame_num);
> > +
> > +	addr_offset = CQ_ADDRESS_OFFSET * (frame_num % CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, isp_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(isp_dev->dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_num:%d cq_addr:0x%x",
> > +		cq_num, frame_num, addr_offset);
> > +
> > +	return cq_num;
> > +}
> > +
> > +static void irq_handle_notify_event(struct isp_device *isp_dev,
> > +				    unsigned int irq_status,
> > +				    unsigned int dma_status,
> > +				    bool sof_only)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned long flags;
> > +	int i;
> > +
> > +	if (irq_status & VS_INT_ST) {
> > +		/* Notify specific HW events to user space */
> > +		mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> > +					     isp_dev->current_frame);
> 
> Shouldn't we call this at SOF_INT_ST and not VS? At least according to the
> definition of the V4L2_EVENT_FRAME_SYNC event at
> https://www.kernel.org/doc/html/latest/media/uapi/v4l/vidioc-dqevent.html
> 

Fix in next patch.
We will change to use SOF_INT_ST to avoid misunderstanding.

> > +		dev_dbg(dev,
> > +			"frame sync is sent:%d:%d\n",
> > +			isp_dev->sof_count,
> > +			isp_dev->current_frame);
> > +		if (sof_only)
> > +			return;
> 
> If this function can be called only to perform this block, perhaps it should
> be split into two functions?
> 
> Also, what happens if we get sof_only, but we don't get VS_INT_ST set in
> irq_status? Is it expected that in such case the other part of the function
> is executed?
> 

Ok, we will call mtk_cam_dev_event_frame_sync function when receiving
SOF_INT_ST ISR event in the caller and remove this block.

> > +	}
> > +
> > +	if (irq_status & SW_PASS1_DON_ST) {
> > +		/* Notify TX thread to send if TX frame is blocked */
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +	}
> > +
> > +	spin_lock_irqsave(&isp_ctx->irq_dequeue_lock, flags);
> > +	i = atomic_read(&isp_ctx->irq_data_end);
> > +	isp_ctx->irq_event_datas[i].frame_seq_no = isp_dev->current_frame;
> > +	isp_ctx->irq_event_datas[i].meta0_vb2_index = isp_dev->meta0_vb2_index;
> > +	isp_ctx->irq_event_datas[i].meta1_vb2_index = isp_dev->meta1_vb2_index;
> > +	isp_ctx->irq_event_datas[i].irq_status_mask =
> > +		(irq_status & INT_ST_MASK_CAM);
> > +	isp_ctx->irq_event_datas[i].dma_status_mask =
> > +		(dma_status & DMA_ST_MASK_CAM);
> > +	atomic_set(&isp_ctx->irq_data_end, ++i & 0x3);
> > +	spin_unlock_irqrestore(&isp_ctx->irq_dequeue_lock, flags);
> > +
> > +	wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> 
> I can see that all isp_deque_work() does is returning buffers to vb2. I
> don't think we need this intricate system to do that, as we could just do
> it inside the interrupt handler, in isp_irq_cam() directly.
> 

Ok, we will move all dequeue function in the ISR function and remove
this dequeue thread and related codes.

> > +
> > +	dev_dbg(dev,
> > +		"%s IRQ:0x%x DMA:0x%x seq:%d idx0:%d idx1:%d\n",
> > +		__func__,
> > +		(irq_status & INT_ST_MASK_CAM),
> > +		(dma_status & DMA_ST_MASK_CAM),
> > +		isp_dev->current_frame,
> > +		isp_dev->meta0_vb2_index,
> > +		isp_dev->meta1_vb2_index);
> > +}
> > +
> > +irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct isp_device *isp_dev = (struct isp_device *)data;
> > +	struct isp_p1_device *p1_dev = get_p1_device(isp_dev->dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = isp_dev->dev;
> > +	unsigned int cam_idx, cq_num, hw_frame_num;
> > +	unsigned int meta0_vb2_index, meta1_vb2_index;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned int aao_fbc, afo_fbc;
> > +	unsigned long flags;
> > +
> > +	/* Check the streaming is off or not */
> > +	if (!p1_dev->cam_dev.streaming)
> > +		return IRQ_HANDLED;
> 
> This shouldn't be needed. The driver needs to unmask the interrupts in
> hardware registers when it starts streaming and mask them when it stops.
> Note that I mean the P1 hardware registers, not disable_irq(), which
> shouldn't be used.
> 

Fix in next patch.

> > +
> > +	cam_idx = isp_dev->isp_hw_module - ISP_CAM_A_IDX;
> > +	cq_num = 0;
> > +
> > +	spin_lock_irqsave(&isp_dev->spinlock_irq, flags);
> > +	irq_status = readl(isp_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	dma_status = readl(isp_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	hw_frame_num = readl(isp_dev->regs + REG_HW_FRAME_NUM);
> > +	meta0_vb2_index = readl(isp_dev->regs + REG_META0_VB2_INDEX);
> > +	meta1_vb2_index = readl(isp_dev->regs + REG_META1_VB2_INDEX);
> 
> Hmm, reading vb2 buffer index from hardware registers? Was this hardware
> designed exclusively for V4L2? ;)
> 
> Jokes aside, how does the hardware know V4L2 buffer indexes?
> 

This is explained in the above.

> > +	aao_fbc = readl(isp_dev->regs + REG_AAO_FBC_STATUS);
> > +	afo_fbc = readl(isp_dev->regs + REG_AFO_FBC_STATUS);
> > +	spin_unlock_irqrestore(&isp_dev->spinlock_irq, flags);
> > +
> > +	/* Ignore unnecessary IRQ */
> > +	if (!irq_status && (!(dma_status & DMA_ST_MASK_CAM)))
> > +		return IRQ_HANDLED;
> 
> Unnecessary IRQs should be masked in the hardware IRQ mask registers. If we
> get an interrupt without any unmasked hardware IRQs active in the status,
> that's an error somewhere and we should return IRQ_NONE.
> 

Ok, we will check the IRQ EN register firstly and check any unmasked
IRQs for IRQ_NONE case.

> > +
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +
> > +	/* Sof, done order check */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > +
> > +		/* Notify IRQ event and enqueue frame */
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > +		isp_dev->current_frame = hw_frame_num;
> 
> What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> event?
> 

This is a another spare register for frame sequence number usage.
It comes from struct p1_frame_param:frame_seq_no which is sent by
SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
Is it a better understanding?

Below is our frame request handling in current design.

1. Buffer preparation
- Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
other meta histogram buffers (LCSO/LMVO) into one request.
- Accumulated one unique frame sequence number to each request and send
this request to the SCP composer to compose CQ (Command queue) buffer
via SCP_ISP_FRAME IPI command.
- CQ buffer is frame registers set. If ISP registers should be updated
per frame, these registers are configured in the CQ buffer, such as
frame sequence number, DMA addresses and tuning ISP registers.
- One frame request will be composed into one CQ buffer.Once CQ buffer
is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
its corresponding frame sequence number. Based on this, kernel driver
knows which request is ready to be en-queued and save this with
p1_dev->isp_ctx.composed_frame_id.
- The maximum number of CQ buffers in SCP is 3.

2. Buffer en-queue flow
- In order to configure correct CQ buffer setting before next SQF event,
it is depended on by MTK ISP P1 HW CQ mechanism.
- The basic concept of CQ mechanism is loaded ISP CQ buffer settings
when HW_PASS1_DON_ST is received which means DMA output is done.
- Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
address is used. Otherwise, it will loaded one dummy CQ buffer to
bypass.
- So we will check available CQ buffers by comparing composed frame
sequence number & dequeued frame sequence from ISP HW in SOF event.
- If there are available CQ buffers, update the CQ base address to the
next CQ buffer address based on current de-enqueue frame sequence
number. So MTK ISP P1 HW will load this CQ buffer into HW when
HW_PASS1_DON_ST is triggered which is before the next SOF.
- So in next SOF event, ISP HW starts to output DMA buffers with this
request until request is done.
- But, for the first request, it is loaded into HW manually when
streaming is on for better performance.

3. Buffer de-queue flow
- We will use frame sequence number to decide which request is ready to
de-queue.
- We will save some important register setting from ISP HW when SOF is
received. This is because the ISP HW starts to output the data with the
corresponding settings, especially frame sequence number setting.
- When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
done. So we could call isp_deque_request_frame with frame sequence
number to de-queue frame to VB2
- For AAO/AFO buffers, it has similar design concept. Sometimes, if only
AAO/AFO non-request buffers are en-queued without request buffers at the
same time, there will be no SW P1 done event for AAO/AFO DMA done.
Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
- Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
we may try to send another request to SCP for composing.

Hopefully, my explanation is helpful for better understanding our
implementation. If you still have any questions, please let me know. 

> > +		isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +		isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +	} else {
> > +		if (irq_status & SOF_INT_ST) {
> > +			isp_dev->current_frame = hw_frame_num;
> > +			isp_dev->meta0_vb2_index = meta0_vb2_index;
> > +			isp_dev->meta1_vb2_index = meta1_vb2_index;
> > +		}
> > +		irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > +	}
> 
> The if and else blocks do almost the same things just in different order. Is
> it really expected?
> 

If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
the correct sequence should be handle HW_PASS1_DON_ST firstly to check
any de-queued frame and update the next frame setting later.
Normally, this is a corner case or system performance issue.

Btw, we will revise the above implementation as below.


if (irq_status & SOF_INT_ST)
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
					     dequeue_frame_seq_no);

/* Sof, done order check */
if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
	dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

/* Notify IRQ event and de-enqueue frame */
irq_handle_notify_event(p1_dev, irq_status, dma_status);

/* Update frame settings & CQ address for frame en-queue */
enqueue_frame_seq_no = 0;
if (irq_status & SOF_INT_ST)
	enqueue_frame_seq_no = irq_handle_sof(p1_dev,
					      dequeue_frame_seq_no,
					      meta0_vb2_index,
					      meta1_vb2_index); 

> > +
> > +	if (irq_status & SOF_INT_ST)
> > +		cq_num = irq_handle_sof(isp_dev, isp_ctx->scp_mem_iova,
> > +					hw_frame_num);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev,
> > +			"raw_int_err:0x%x/0x%x\n",
> > +			irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_dump_dma_status(isp_dev);
> > +	}
> > +
> > +	if (irq_status & INT_ST_LOG_MASK_CAM)
> > +		dev_dbg(dev, IRQ_STAT_STR,
> 
> Please just put that string here, otherwise the reader would have no idea
> what message is being printed here.
> 

Fix in next patch.

> > +			'A' + cam_idx,
> > +			isp_dev->sof_count,
> > +			irq_status,
> > +			dma_status,
> > +			hw_frame_num,
> > +			cq_num,
> > +			aao_fbc,
> > +			afo_fbc);
> 
> nit: No need to put each argument in its own line.
> 

Fix in next patch.

> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct isp_p1_device *p1_dev)
> > +{
> > +	phandle rproc_phandle;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	p1_dev->scp_pdev = scp_get_pdev(p1_dev->pdev);
> > +	if (!p1_dev->scp_pdev) {
> > +		dev_err(dev, "Failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
> > +				   &rproc_phandle);
> > +	if (ret) {
> > +		dev_err(dev, "fail to get rproc_phandle:%d\n", ret);
> > +		return -EINVAL;
> > +	}
> > +
> > +	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n\n", p1_dev->rproc_handle);
> > +	if (!p1_dev->rproc_handle) {
> > +		dev_err(dev, "fail to get rproc_handle\n");
> > +		return -EINVAL;
> > +	}
> 
> This look-up should be done once in probe(). Only the rproc_boot() should
> happen dynamically.
> 

Fix in next patch.

> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		/*
> > +		 * Return 0 if downloading firmware successfully,
> > +		 * otherwise it is failed
> > +		 */
> > +		return -ENODEV;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int isp_init_context(struct isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	unsigned int i;
> > +
> > +	dev_dbg(dev, "init irq work thread\n");
> > +	if (!isp_ctx->isp_deque_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread =
> > +			kthread_run(isp_deque_work, (void *)p1_dev,
> > +				    "isp_deque_work");
> > +		if (IS_ERR(isp_ctx->isp_deque_thread.thread)) {
> > +			dev_err(dev, "unable to alloc kthread\n");
> > +			isp_ctx->isp_deque_thread.thread = NULL;
> > +			return -ENOMEM;
> > +		}
> > +	}
> > +	spin_lock_init(&isp_ctx->irq_dequeue_lock);
> > +	mutex_init(&isp_ctx->lock);
> > +
> > +	INIT_LIST_HEAD(&isp_ctx->p1_enqueue_list.queue);
> > +	atomic_set(&isp_ctx->p1_enqueue_list.queue_cnt, 0);
> > +
> > +	for (i = 0; i < ISP_DEV_NODE_NUM; i++)
> > +		spin_lock_init(&p1_dev->isp_devs[i].spinlock_irq);
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +
> > +	atomic_set(&isp_ctx->irq_data_end, 0);
> > +	atomic_set(&isp_ctx->irq_data_start, 0);
> > +
> > +	return 0;
> 
> Everything here looks like something that should be done once in probe. I
> also don't see a point of having a separate mtk_isp_p1_ctx struct for the
> data above. It could be just located in p1_dev, at least for now.
> 
> If we end up implementing support for multiple contexts, we could isolate
> per-context data then, once we know what's really per-context. For now,
> let's keep it simple.
> 

Ok, we will remove isp_ctx structure and move some fields into p1_dev or
cam_dev,

> > +}
> > +
> > +static int isp_uninit_context(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct mtk_isp_queue_job *framejob, *tmp_framejob;
> > +
> > +	spin_lock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +	list_for_each_entry_safe(framejob, tmp_framejob,
> > +				 &isp_ctx->p1_enqueue_list.queue, list_entry) {
> > +		list_del(&framejob->list_entry);
> > +		kfree(framejob);
> > +	}
> > +	spin_unlock_irq(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	if (isp_ctx->isp_deque_thread.thread) {
> > +		kthread_stop(isp_ctx->isp_deque_thread.thread);
> > +		wake_up_interruptible(&isp_ctx->isp_deque_thread.wq);
> > +		isp_ctx->isp_deque_thread.thread = NULL;
> > +	}
> > +
> > +	mutex_destroy(&isp_ctx->lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned int get_enabled_dma_ports(struct mtk_cam_dev *cam_dev)
> > +{
> > +	unsigned int enabled_dma_ports, i;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	enabled_dma_ports = 0;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		if (cam_dev->vdev_nodes[i].enabled)
> > +			enabled_dma_ports |=
> > +				cam_dev->vdev_nodes[i].desc.dma_port;
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s :0x%x", __func__, enabled_dma_ports);
> > +
> > +	return enabled_dma_ports;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return RAW_PXL_ID_R;
> > +	default:
> 
> Could we fail here instead?
> 

Ok, we will return invalid value here, like MTK_CAM_RAW_PXL_ID_UNKNOWN.
We will also check this error in the caller.

> > +		return RAW_PXL_ID_B;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return IMG_FMT_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +		return IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +		return IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +		return IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +		return IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_pixel_byte(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_B8:
> > +	case V4L2_PIX_FMT_MTISP_F8:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_B10:
> > +	case V4L2_PIX_FMT_MTISP_F10:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_B12:
> > +	case V4L2_PIX_FMT_MTISP_F12:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_B14:
> > +	case V4L2_PIX_FMT_MTISP_F14:
> > +		return 14;
> > +	default:
> 
> Could we fail here instead, so that we don't mask some potential errors?
> 

Ok, we will return MTK_CAM_IMG_FMT_UNKNOWN and check this error in the
caller.

> > +		return 10;
> > +	}
> > +}
> > +
> > +static void config_img_fmt(struct device *dev, struct p1_img_output *out_fmt,
> > +			   const struct v4l2_format *in_fmt,
> > +			   const struct v4l2_subdev_format *sd_format)
> > +{
> > +	out_fmt->img_fmt = get_img_fmt(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_byte = get_pixel_byte(in_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->size.w = in_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = in_fmt->fmt.pix_mp.height;
> > +
> > +	out_fmt->size.stride = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = in_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> 
> Please group operations on the same field together, i.e. remove the blank
> line above size.stride and add one blank line above size.w.
> 

Fix in next patch.

> > +
> > +	out_fmt->crop.left = 0x0;
> > +	out_fmt->crop.top = 0x0;
> > +
> 
> Remove the blank line.
> 

Fix in next patch.

> > +	out_fmt->crop.width = sd_format->format.width;
> > +	out_fmt->crop.height = sd_format->format.height;
> > +
> > +	WARN_ONCE(in_fmt->fmt.pix_mp.width > out_fmt->crop.width ||
> > +		  in_fmt->fmt.pix_mp.height > out_fmt->crop.height,
> > +		  "img out:%d:%d in:%d:%d",
> > +		  in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height,
> > +		  out_fmt->crop.width, out_fmt->crop.height);
> 
> We should check this earlier and fail the streaming start if there is a
> mismatch between sensor and video node configuration.
> 

Fix in next patch.

> > +
> > +	dev_dbg(dev, "pixel_byte:%d img_fmt:0x%x\n",
> > +		out_fmt->pixel_byte,
> > +		out_fmt->img_fmt);
> > +	dev_dbg(dev,
> > +		"param:size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +}
> > +
> > +/* ISP P1 interface functions */
> > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > +{
> > +	struct device *dev = &cam_dev->pdev->dev;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = isp_init_context(p1_dev);
> > +	if (ret)
> > +		return ret;
> 
> The above function doesn't really seem to be related to power management.
> Should it be called from subdev stream on?
> 

We will rename this function to mtk_isp_hw_init.
But, it will be called when the first video node is streamed on.
This is because we need to initialize the HW firstly for sub-device
stream-on performance.  We need to send some IPI commands, such as
ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
timing.

> > +
> > +	ret = isp_composer_init(dev);
> > +	if (ret)
> > +		goto composer_err;
> 
> This also doesn't seem to be related to power management.
> 

Ok, we will rename to mtk_isp_hw_init to avoid misunderstanding.

> > +
> > +	pm_runtime_get_sync(dev);
> > +
> > +	/* ISP HW INIT */
> > +	isp_ctx->isp_hw_module = ISP_CAM_B_IDX;
> 
> There is some amount of code handling various values of isp_hw_module in
> this driver. If we're hardcoding ISP_CAM_B_IDX here, it's basically dead
> code that can't be tested. Please either add support for all the indexes in
> the driver or simplify all the code to just handle CAM_B.
> 

Ok, we will simplify driver code to support single CAM only.
We will remove isp_hw_module usage in the source code.

> > +	/* Use pure RAW as default HW path */
> > +	isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > +	atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > +
> > +	isp_composer_hw_init(dev);
> > +	/* Check enabled DMAs which is configured by media setup */
> > +	isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> 
> Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> necessary?
> 

Yes, it is necessary for non-request buffers design for Camera 3A video
nodes. For 3A video nodes, we just want to know which 3A video nodes are
enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
format from user space. So we just pass the enabled DMA information from
kernel to SCP only. With 3A enabled DMA information, we could configure
related 3A registers in SCP.

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +
> > +composer_err:
> > +	isp_uninit_context(dev);
> > +
> > +	return ret;
> > +}
> > +
> > +int mtk_isp_power_release(struct device *dev)
> > +{
> > +	isp_composer_hw_deinit(dev);
> > +	isp_uninit_context(dev);
> 
> These two don't seem to be related to power either.
> 
> Instead, I don't see anything that could undo the rproc_boot() operation
> here.
> 

We will rename this function to mtk_isp_hw_release.
We will also add rproc_shutdown function call here.

int mtk_isp_hw_release(struct mtk_cam_dev *cam)
{
	struct device *dev = cam->dev;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	isp_composer_hw_deinit(p1_dev);
	pm_runtime_put_sync_autosuspend(dev);
	rproc_shutdown(p1_dev->rproc_handle);

	dev_dbg(dev, "%s done\n", __func__);

	return 0;
}

> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_config(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_config_param config_param;
> > +	struct mtk_cam_dev *cam_dev = &p1_dev->cam_dev;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	unsigned int enabled_dma_ports;
> > +	struct v4l2_format *img_fmt;
> > +	int ret;
> > +
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].current_frame = 0;
> > +	p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count = 0;
> > +
> > +	isp_ctx->frame_seq_no = 1;
> > +	atomic_set(&isp_ctx->composed_frame_id, 0);
> > +
> > +	/* Get the enabled DMA ports */
> > +	enabled_dma_ports = get_enabled_dma_ports(cam_dev);
> > +	dev_dbg(dev, "%s enable_dma_ports:0x%x", __func__, enabled_dma_ports);
> > +
> > +	/* Sensor config */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam_dev->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +
> 
> Unnecessary blank line.
> 

Fix in next patch.

> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt on failed:%d\n", ret);
> > +		return -EPERM;
> 
> return ret?
> 

Fix in next patch.

> > +	}
> > +
> > +	dev_dbg(dev,
> > +		"get_fmt ret=%d, w=%d, h=%d, code=0x%x, field=%d, color=%d\n",
> > +		ret, sd_fmt.format.width, sd_fmt.format.height,
> > +		sd_fmt.format.code, sd_fmt.format.field,
> > +		sd_fmt.format.colorspace);
> > +
> > +	config_param.cfg_in_param.continuous = 0x1;
> > +	config_param.cfg_in_param.subsample = 0x0;
> > +	/* Fix to one pixel mode in default */
> > +	config_param.cfg_in_param.pixel_mode = 0x1;
> > +	/* Support normal pattern in default */
> > +	config_param.cfg_in_param.data_pattern = 0x0;
> > +
> > +	config_param.cfg_in_param.crop.left = 0x0;
> > +	config_param.cfg_in_param.crop.top = 0x0;
> 
> Why hexadecimal numerals? Also, what's the meaning of these values? For
> anything boolean, you could just use true and false as a substitute of 0 and
> 1. For anything that has more values, please define the values using macros.
> 

Fix in next patch.
1. Remove unnecessary hexadecimal number usage
2. Use boolean value if possible
3. Assign non-empty field values.

Below is the fixed version.

	config_param.cfg_in_param.continuous = true;
	/* Fix to one pixel mode in default */
	config_param.cfg_in_param.pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
	config_param.cfg_in_param.raw_pixel_id =
		get_sensor_pixel_id(sd_fmt.format.code);
	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
	if (config_param.cfg_in_param.img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
	    config_param.cfg_in_param.raw_pixel_id ==
MTK_CAM_RAW_PXL_ID_UNKNOWN) {
		dev_err(dev, "Unknown cfg img fmt or unknown raw pixel id\n");
		return -EINVAL;
	}

> > +
> > +	config_param.cfg_in_param.raw_pixel_id =
> > +		get_sensor_pixel_id(sd_fmt.format.code);
> > +	config_param.cfg_in_param.img_fmt = get_sensor_fmt(sd_fmt.format.code);
> > +	config_param.cfg_in_param.crop.width = sd_fmt.format.width;
> > +	config_param.cfg_in_param.crop.height = sd_fmt.format.height;
> 
> Move the other crop settings from above to here.
> 

Ditto.

> > +
> > +	config_param.cfg_main_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_MAIN_STREAM_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_IMGO) == R_IMGO) {
> 
> No need for the == R_IMGO part.
> 

Since R_IMGO is mandatory video node, we will remove this checking.

> > +		config_param.cfg_main_param.bypass = 0;
> > +		config_param.cfg_main_param.pure_raw = isp_ctx->isp_raw_path;
> > +		config_param.cfg_main_param.pure_raw_pack = 1;
> > +		config_img_fmt(dev, &config_param.cfg_main_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	config_param.cfg_resize_param.bypass = 1;
> > +	img_fmt = &cam_dev->vdev_nodes[MTK_CAM_P1_PACKED_BIN_OUT].vdev_fmt;
> > +	if ((enabled_dma_ports & R_RRZO) == R_RRZO) {
> 
> Ditto.
> 

Fix in next patch as below:

if (enabled_dma_ports & R_RRZO) {
	ret = config_img_fmt(cam,
			     &config_param.cfg_resize_param.output,
			     img_fmt, &sd_fmt);
	if (ret)
		return ret;
} else {
	config_param.cfg_resize_param.bypass = true;
}

> > +		config_param.cfg_resize_param.bypass = 0;
> > +		config_img_fmt(dev, &config_param.cfg_resize_param.output,
> > +			       img_fmt, &sd_fmt);
> > +	}
> > +
> > +	/* Configure meta DMAs info. */
> > +	config_param.cfg_meta_param.enabled_meta_dmas = enabled_dma_ports;
> 
> Should image DMAs be masked out of this bitmap?
> 

We will replace struct cfg_meta_param with enabled_dmas integer.
So we can pass all enabled DMA masks to SCP. 

> > +
> > +	isp_composer_hw_config(dev, &config_param);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_enqueue(struct device *dev, unsigned int dma_port,
> > +		     struct mtk_cam_dev_buffer *buffer)
> > +{
> > +	struct mtk_isp_scp_p1_cmd frameparams;
> > +
> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	frameparams.cmd_id = ISP_CMD_ENQUEUE_META;
> > +	frameparams.meta_frame.enabled_dma = dma_port;
> > +	frameparams.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > +	frameparams.meta_frame.meta_addr.iova = buffer->daddr;
> > +	frameparams.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_CMD);
> > +}
> > +
> > +void mtk_isp_req_flush_buffers(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_queue_job *job, *j0;
> > +	struct mtk_cam_dev_buffer *buf, *b0;
> > +	struct isp_queue *p1_list = &p1_dev->isp_ctx.p1_enqueue_list;
> > +
> > +	if (!atomic_read(&p1_list->queue_cnt))
> > +		return;
> 
> Do we need this explicit check? The code below wouldn't do anything if there
> isn't anything in the list. IMHO we could even remove queue_cnt completely.
> 

Since we have redesign request en-queue mechanism, this function will be
removed.

> > +
> > +	spin_lock(&p1_list->lock);
> > +	list_for_each_entry_safe(job, j0, &p1_list->queue, list_entry) {
> 
> nit: s/j0/job_prev/
> 

Will apply this naming rule in next patch.

> > +		list_for_each_entry_safe(buf, b0, &job->list_buf, list) {
> 
> nit: s/b0/buf_pref/
> 

Ditto.

> Also, we should be able to replace this with iterating over the generic list
> of request objects, rather than this internal list.
> 
> > +			list_del(&buf->list);
> > +			if (buf->vbb.vb2_buf.state == VB2_BUF_STATE_ACTIVE)
> 
> It shouldn't be possible to happen. If you see this check failing, that
> means a problem somewhere else in the driver.
> 

Fix in next patch.

> > +				vb2_buffer_done(&buf->vbb.vb2_buf,
> > +						VB2_BUF_STATE_ERROR);
> > +		}
> > +		list_del(&job->list_entry);
> > +		atomic_dec(&p1_list->queue_cnt);
> > +		kfree(job);
> > +	}
> > +	spin_unlock(&p1_list->lock);
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

We got your point.
Below is new function declaration.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	struct p1_frame_param frameparams;
> > +	struct mtk_isp_queue_job *framejob;
> > +	struct media_request_object *obj, *obj_safe;
> > +	struct vb2_buffer *vb;
> > +	struct mtk_cam_dev_buffer *buf;
> > +
> > +	framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> 
> This allocation shouldn't be needed. The structure should be already a part
> of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> even say that the contents of this struct should be just moved to that one
> to avoid overabstracting.
> 

For this function, we will apply the new design from P2 driver's
comment. Here is the new implementation.

struct mtk_cam_dev_request {
	struct media_request req;
	struct mtk_p1_frame_param frame_params;
	struct work_struct frame_work;
	struct list_head list;
	atomic_t buf_count;
};

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

> > +	memset(framejob, 0, sizeof(*framejob));
> 
> Putting the above comment aside, kzalloc() already zeroes the memory.
> 

Ditto.

> > +	memset(&frameparams, 0, sizeof(frameparams));
> > +	INIT_LIST_HEAD(&framejob->list_buf);
> > +
> > +	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;
> > +	frameparams.sof_idx =
> > +		p1_dev->isp_devs[isp_ctx->isp_hw_module].sof_count;
> 
> How is this synchronized with the sof_count increment in irq_handle_sof()?
> 

The sof_idx field is only used for debugging purpose.
We will remove this.

> > +	framejob->frame_seq_no = frameparams.frame_seq_no;
> > +
> > +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> 
> We should check that the object type before assuming it's a vb2_buffer by
> calling vb2_request_object_is_buffer().
> 

Fix in next patch.

> > +		buf = container_of(vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +		framejob->request_fd = buf->vbb.request_fd;
> 
> We shouldn't use request_fd as the key here. We already have req here, which
> is the right key to use.
> 
> That said, I can see framejob->request_fd only used for printing a debugging
> message in mtk_cam_dev_job_finish(). The request API core should already
> print something for us once a request is completed, so perhaps that isn't
> needed?
> 

Fix in next patch.

> > +		frameparams.dma_buffers[buf->node_id].iova = buf->daddr;
> > +		frameparams.dma_buffers[buf->node_id].scp_addr = buf->scp_addr;
> > +		list_add_tail(&buf->list, &framejob->list_buf);
> 
> Why do we need this private list? We could just call exactly the same
> list_for_each() over the request objects.
> 

This function is re-designed.

> > +	}
> > +
> > +	spin_lock(&isp_ctx->p1_enqueue_list.lock);
> > +	list_add_tail(&framejob->list_entry, &isp_ctx->p1_enqueue_list.queue);
> 
> We already have a list head in mtk_cam_dev_request.
> 

This function is re-designed.

> > +	atomic_inc(&isp_ctx->p1_enqueue_list.queue_cnt);
> > +	spin_unlock(&isp_ctx->p1_enqueue_list.lock);
> > +
> > +	isp_composer_enqueue(dev, &frameparams, SCP_ISP_FRAME);
> > +	dev_dbg(dev, "request fd:%d frame_seq_no:%d is queued cnt:%d\n",
> > +		framejob->request_fd,
> > +		frameparams.frame_seq_no,
> > +		atomic_read(&isp_ctx->p1_enqueue_list.queue_cnt));
> > +}
> > +
> > +static int enable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	int ret;
> > +
> > +	dev_info(dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +
> > +	ret = clk_bulk_prepare_enable(p1_dev->isp_ctx.num_clks,
> > +				      p1_dev->isp_ctx.clk_list);
> > +	if (ret)
> > +		goto clk_err;
> > +
> > +	return 0;
> > +
> > +clk_err:
> > +	dev_err(dev, "cannot pre-en isp_cam clock:%d\n", ret);
> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> 
> clk_bulk_prepare_enable() returns without any clocks enabled if it fails, so
> this would disable the clocks second time.
> 

Fix in next patch.

> > +	return ret;
> > +}
> > +
> > +static void disable_sys_clock(struct isp_p1_device *p1_dev)
> > +{
> > +	dev_info(&p1_dev->pdev->dev, "- %s\n", __func__);
> 
> dev_dbg()
> 

Fix in next patch.

> > +	clk_bulk_disable_unprepare(p1_dev->isp_ctx.num_clks,
> > +				   p1_dev->isp_ctx.clk_list);
> > +}
> 
> There is no point in having wrapper functions to just call one function
> inside. Please just call clk_bulk_*() directly.
> 

Fix in next patch.

> > +
> > +static int mtk_isp_suspend(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check if the device isn't already runtime suspended. If it is, we
> don't have to do anything here and can just return.
> 
> 

Add pm_runtime_suspended(dev) to check.

> We also need to ensure that no new requests are queued to the hardware at
> this point. This could be done by replacing any of the kthreads with
> workqueues and making all of the workqueues freezable.
> 

Yes, we will use workqueue to send frame request.
Here is the workqueue's definition.

	p1_dev->composer_wq =
		alloc_ordered_workqueue(dev_name(p1_dev->dev),
					__WQ_LEGACY | WQ_MEM_RECLAIM |
					WQ_FREEZABLE);

> > +	isp_dev = &p1_dev->isp_devs[module];
> > +	reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +	if (reg_val & VFDATA_EN_BIT) {
> > +		dev_dbg(dev, "Cam:%d suspend, disable VF\n", module);
> > +		/* Disable view finder */
> > +		writel((reg_val & (~VFDATA_EN_BIT)),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +		/*
> > +		 * After VF enable, the TG frame count will be reset to 0;
> > +		 */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val & (~CMOS_EN_BIT)),
> > +		       isp_dev->regs +  + REG_TG_SEN_MODE);
> > +	}
> 
> Are you sure this is the right handling? We need to make sure the hardware
> finishes processing the current frame and stops.
> 

We will revise this handling to make sure the ISP HW is idle before
suspend.

> > +
> > +	disable_sys_clock(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_resume(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	int module = p1_dev->isp_ctx.isp_hw_module;
> > +	struct isp_device *isp_dev = &p1_dev->isp_devs[module];
> > +	unsigned int reg_val;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> 
> We need to check runtime PM status here as well, because if the device was
> suspended, there is nothing to do here.
> 

Ditto.

> > +	enable_sys_clock(p1_dev);
> > +
> > +	/* V4L2 stream-on phase & restore HW stream-on status */
> > +	if (p1_dev->cam_dev.streaming) {
> > +		dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > +		/* Enable CMOS */
> > +		reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > +		writel((reg_val | CMOS_EN_BIT),
> > +		       isp_dev->regs + REG_TG_SEN_MODE);
> > +		/* Enable VF */
> > +		reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > +		writel((reg_val | VFDATA_EN_BIT),
> > +		       isp_dev->regs + REG_TG_VF_CON);
> > +	}
> 
> Does the hardware keep all the state, e.g. queued buffers, during suspend?
> Would the code above continue all the capture from the next buffer, as
> queued by the userspace before the suspend?
> 

Yes, we will test it.
1. SCP buffers are kept by SCP processor
2. ISP registers are still kept even if ISP clock is disable.

> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	struct isp_p1_device *p1_dev;
> > +	struct mtk_isp_p1_ctx *isp_ctx;
> > +	struct isp_device *isp_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq;
> > +	int ret;
> > +	unsigned int i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	dev_set_drvdata(dev, p1_dev);
> > +	isp_ctx = &p1_dev->isp_ctx;
> > +	p1_dev->pdev = pdev;
> 
> Perhaps you want to store &pdev->dev instead of pdev? I'm not sure a
> reference to platform_device is very useful later in the code.
> 

Fix in next patch.

> > +
> > +	for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> 
> I think we want to start from 0 here?
> 

Because of single CAM support, we will revise our DTS tree to support
single CAM only. So we could remove this loop and check the CAM-B HW
information here. Here is below new function.

static int mtk_isp_probe(struct platform_device *pdev)
{
	/* List of clocks required by isp cam */
	static const char * const clk_names[] = {
		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
	};
	struct mtk_isp_p1_device *p1_dev;
	struct device *dev = &pdev->dev;
	struct resource *res;
	int irq, ret, i;

	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
	if (!p1_dev)
		return -ENOMEM;

	p1_dev->dev = dev;
	dev_set_drvdata(dev, p1_dev);

	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
	p1_dev->regs = devm_ioremap_resource(dev, res);
	if (IS_ERR(p1_dev->regs)) {
		dev_err(dev, "Failed platform resources map\n");
		return PTR_ERR(p1_dev->regs);
	}
	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);

	irq = platform_get_irq(pdev, 0);
	if (!irq) {
		dev_err(dev, "Missing IRQ resources data\n");
		return -ENODEV;
	}
	ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
			       dev_name(dev), p1_dev);
	if (ret) {
		dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
			dev->of_node->name, irq);
		return ret;
	}
	dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
	spin_lock_init(&p1_dev->spinlock_irq);

	p1_dev->num_clks = ARRAY_SIZE(clk_names);
	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
				    sizeof(*p1_dev->clks), GFP_KERNEL);
	if (!p1_dev->clks)
		return -ENOMEM;

	for (i = 0; i < p1_dev->num_clks; ++i)
		p1_dev->clks[i].id = clk_names[i];

	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
		return ret;
	}

	ret = isp_setup_scp_rproc(p1_dev, pdev);
	if (ret)
		return ret;

	pm_runtime_enable(dev);

	/* Initialize the v4l2 common part */
	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
	if (ret)
		return ret;

	return 0;
}

> > +		isp_dev = &p1_dev->isp_devs[i];
> > +		isp_dev->isp_hw_module = i;
> > +		isp_dev->dev = dev;
> 
> p1_dev already includes a pointer to this.
> 

Fix in next patch.

> > +		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > +		isp_dev->regs = devm_ioremap_resource(dev, res);
> > +
> > +		dev_dbg(dev, "cam%u, map_addr=0x%lx\n",
> > +			i, (unsigned long)isp_dev->regs);
> > +
> > +		if (!isp_dev->regs)
> 
> devm_ioremap_resource() returns ERR_PTR() not NULL on error.
> 

Fix in next patch.

> > +			return PTR_ERR(isp_dev->regs);
> > +
> > +		/* Support IRQ from ISP_CAM_A_IDX */
> > +		if (i >= ISP_CAM_A_IDX) {
> > +			/* Reg & interrupts index is shifted with 1  */
> 
> The reader can already see that it's shifted from the code below. The
> comment should say _why_ it is shifted.
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			irq = platform_get_irq(pdev, i - 1);
> 
> The bindings have 3 IRQs, but we only seem to request 2 here. Is that
> expected?
> 

ok, we will remove this checking after supporting single CAM HW.

> > +			if (irq) {
> 
> Please invert this if, so that it bails out on error. Also, this should
> check for negative errors codes, not 0.
> 

Fix in next patch.

> > +				ret = devm_request_irq(dev, irq,
> > +						       isp_irq_cam,
> > +						       IRQF_SHARED,
> > +						       dev_driver_string(dev),
> 
> Use dev_name().
> 

Fix in next patch.

> > +						       (void *)isp_dev);
> 
> No need to cast to void *.
> 

Fix in next patch.

> > +				if (ret) {
> > +					dev_err(dev,
> > +						"req_irq fail, dev:%s irq=%d\n",
> > +						dev->of_node->name,
> > +						irq);
> > +					return ret;
> > +				}
> > +				dev_dbg(dev, "Registered irq=%d, ISR:%s\n",
> > +					irq, dev_driver_string(dev));
> > +			}
> > +		}
> > +		spin_lock_init(&isp_dev->spinlock_irq);
> > +	}
> 
> We might want to move out the body of this loop to a separate function to
> keep this function shorter.
> 

Since we have already remove this loop, maybe we don't need to move out
the body into one single function.

> > +
> > +	p1_dev->isp_ctx.num_clks = ARRAY_SIZE(mtk_isp_clks);
> > +	p1_dev->isp_ctx.clk_list =
> 
> nit: "list" is the term commonly used for the list data structure. There is
> also a convention to call the length of array XXX num_XXX, so how about
> clks and num_clks?
> 

Fix in next patch.

> > +		devm_kcalloc(dev,
> > +			     p1_dev->isp_ctx.num_clks,
> > +			     sizeof(*p1_dev->isp_ctx.clk_list),
> > +			     GFP_KERNEL);
> > +	if (!p1_dev->isp_ctx.clk_list)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->isp_ctx.num_clks; ++i)
> > +		p1_dev->isp_ctx.clk_list->id = mtk_isp_clks[i];
> 
> Shouldn't this be clk_list[i].id?
> 

Fix in next patch.

> > +
> > +	ret = devm_clk_bulk_get(dev,
> > +				p1_dev->isp_ctx.num_clks,
> > +				p1_dev->isp_ctx.clk_list);
> > +	if (ret) {
> > +		dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize reserved DMA memory */
> > +	ret = mtk_cam_reserved_memory_init(p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to configure DMA memory:%d\n", ret);
> > +		goto err_init;
> > +	}
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret)
> > +		goto err_init;
> > +
> > +	spin_lock_init(&isp_ctx->p1_enqueue_list.lock);
> > +	pm_runtime_enable(dev);
> > +
> > +	return 0;
> > +
> > +err_init:
> > +	if (p1_dev->cam_dev.smem_dev)
> > +		device_unregister(p1_dev->cam_dev.smem_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	pm_runtime_disable(dev);
> > +	mtk_cam_dev_release(pdev, &p1_dev->cam_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> 
> For V4L2 drivers system and runtime PM ops would normally be completely
> different. Runtime PM ops would be called when the hardware is idle already
> or is about to become active. System PM ops would be called at system power
> state change and the hardware might be both idle or active. Please also see
> my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> 

Here is the new implementation. It should be clear to show the
difference between system and runtime PM ops. 

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
				pm_runtime_force_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
NULL)
};

> > +};
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam",
> 
> Shouldn't this be something like mtk-cam-p1? Please make sure this
> reasonably consistent with other drivers.
> 

Fix in next patch.

> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Camera ISP driver");
> 
> Mediatek Camera P1 ISP driver? Please make sure this is reasonably
> consistent with other drivers (SenInf, DIP, FD).
> 
> > +MODULE_LICENSE("GPL");
> 
> GPL v2
> 

We will check this naming with other drivers & fix the version.

> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..6af3f569664c
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,243 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef __CAMERA_ISP_H
> > +#define __CAMERA_ISP_H
> > +
> > +#include <linux/cdev.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/ioctl.h>
> > +#include <linux/irqreturn.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/pm_qos.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "mtk_cam-scp.h"
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +#define CAM_A_MAX_WIDTH		3328
> > +#define CAM_A_MAX_HEIGHT		2496
> > +#define CAM_B_MAX_WIDTH		5376
> > +#define CAM_B_MAX_HEIGHT		4032
> > +
> > +#define CAM_MIN_WIDTH			80
> > +#define CAM_MIN_HEIGHT			60
> > +
> > +#define IMG_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define IMG_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define IMG_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define IMG_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define RRZ_MAX_WIDTH			CAM_B_MAX_WIDTH
> > +#define RRZ_MAX_HEIGHT			CAM_B_MAX_HEIGHT
> > +#define RRZ_MIN_WIDTH			CAM_MIN_WIDTH
> > +#define RRZ_MIN_HEIGHT			CAM_MIN_HEIGHT
> > +
> > +#define R_IMGO				BIT(0)
> > +#define R_RRZO				BIT(1)
> > +#define R_AAO				BIT(3)
> > +#define R_AFO				BIT(4)
> > +#define R_LCSO				BIT(5)
> > +#define R_PDO				BIT(6)
> > +#define R_LMVO				BIT(7)
> > +#define R_FLKO				BIT(8)
> > +#define R_RSSO				BIT(9)
> > +#define R_PSO				BIT(10)
> > +
> > +#define CQ_BUFFER_COUNT		3
> > +#define IRQ_DATA_BUF_SIZE		4
> > +#define CQ_ADDRESS_OFFSET		0x640
> > +
> > +#define ISP_COMPOSING_MAX_NUM		4
> > +#define ISP_FRAME_COMPOSING_MAX_NUM	3
> > +
> > +#define IRQ_STAT_STR	"cam%c, SOF_%d irq(0x%x), " \
> > +			"dma(0x%x), frame_num(%d)/cq_num(%d), " \
> > +			"fbc1(0x%x), fbc2(0x%x)\n"
> > +
> > +/*
> > + * In order with the sequence of device nodes defined in dtsi rule,
> > + * one hardware module should be mapping to one node.
> > + */
> > +enum isp_dev_node_enum {
> > +	ISP_CAMSYS_CONFIG_IDX = 0,
> > +	ISP_CAM_UNI_IDX,
> > +	ISP_CAM_A_IDX,
> > +	ISP_CAM_B_IDX,
> > +	ISP_DEV_NODE_NUM
> > +};
> > +
> > +/* Image RAW path for ISP P1 module. */
> > +enum isp_raw_path_enum {
> > +	ISP_PROCESS_RAW_PATH = 0,
> > +	ISP_PURE_RAW_PATH
> > +};
> > +
> > +/* State for struct mtk_isp_p1_ctx: composer_state */
> > +enum  {
> > +	SCP_ON = 0,
> > +	SCP_OFF
> > +};
> 
> Hmm, looks like bool.
> 

This will be removed in next patch.

> > +
> > +enum {
> > +	IMG_FMT_UNKNOWN		= 0x0000,
> > +	IMG_FMT_RAW_START	= 0x2200,
> > +	IMG_FMT_BAYER8		= IMG_FMT_RAW_START,
> > +	IMG_FMT_BAYER10,
> > +	IMG_FMT_BAYER12,
> > +	IMG_FMT_BAYER14,
> > +	IMG_FMT_FG_BAYER8,
> > +	IMG_FMT_FG_BAYER10,
> > +	IMG_FMT_FG_BAYER12,
> > +	IMG_FMT_FG_BAYER14,
> > +};
> > +
> > +enum {
> > +	RAW_PXL_ID_B = 0,
> > +	RAW_PXL_ID_GB,
> > +	RAW_PXL_ID_GR,
> > +	RAW_PXL_ID_R
> > +};
> 
> Please use macros with explicitly assigned values for hardware/firmware ABI
> definitions.
> 

Fix in next patch.

> > +
> > +struct isp_queue {
> > +	struct list_head queue;
> > +	atomic_t queue_cnt;
> > +	spinlock_t lock; /* queue attributes protection */
> > +};
> > +
> > +struct isp_thread {
> > +	struct task_struct *thread;
> > +	wait_queue_head_t wq;
> > +};
> > +
> > +struct mtk_isp_queue_work {
> > +	union {
> > +		struct mtk_isp_scp_p1_cmd cmd;
> > +		struct p1_frame_param frameparams;
> > +	};
> > +	struct list_head list_entry;
> > +	enum mtk_isp_scp_type type;
> > +};
> > +
> > +struct mtk_cam_dev_stat_event_data {
> > +	__u32 frame_seq_no;
> > +	__u32 meta0_vb2_index;
> > +	__u32 meta1_vb2_index;
> > +	__u32 irq_status_mask;
> > +	__u32 dma_status_mask;
> > +};
> > +
> > +struct mtk_isp_queue_job {
> > +	struct list_head list_entry;
> > +	struct list_head list_buf;
> > +	unsigned int request_fd;
> > +	unsigned int frame_seq_no;
> > +};
> 
> Please document the structs above using kerneldoc.
> 

These structures are removed in next patch.

> > +
> > +/*
> > + * struct isp_device - the ISP device information
> > + *
> > + * @dev: Pointer to struct device
> > + * @regs: Camera ISP base register address
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @current_frame: Current frame sequence number, set when SOF
> > + * @meta0_vb2_index: Meta0 vb2 buffer index, set when SOF
> > + * @meta1_vb2_index: Meta1 vb2 buffer index, set when SOF
> > + * @sof_count: The accumulated SOF counter
> > + * @isp_hw_module: Identity camera A or B
> > + *
> > + */
> > +struct isp_device {
> 
> mtk_isp_device?
> 

Fix in next patch.

> > +	struct device *dev;
> > +	void __iomem *regs;
> > +	spinlock_t spinlock_irq; /* ISP reg setting integrity */
> > +	unsigned int current_frame;
> > +	unsigned int meta0_vb2_index;
> > +	unsigned int meta1_vb2_index;
> > +	u8 sof_count;
> > +	u8 isp_hw_module;
> > +};
> > +
> > +/*
> > + * struct mtk_isp_p1_ctx - the ISP device information
> > + *
> > + * @composer_txlist: Queue for SCP TX data including SCP_ISP_CMD & SCP_ISP_FRAME
> > + * @composer_tx_thread: TX Thread for SCP data tranmission
> > + * @cmd_queued: The number of SCP_ISP_CMD commands will be sent
> > + * @ipi_occupied: The total number of SCP TX data has beent sent
> > + * @scp_state: The state of SCP control
> > + * @composing_frame: The total number of SCP_ISP_FRAME has beent sent
> > + * @composed_frame_id: The ack. frame sequence by SCP
> > + * @composer_deinit_thread: The de-initialized thread
> > + * @p1_enqueue_list: Queue for ISP frame buffers
> > + * @isp_deque_thread: Thread for handling ISP frame buffers dequeue
> > + * @irq_event_datas: Ring buffer for struct mtk_cam_dev_stat_event_data data
> > + * @irq_data_start: Start index of irq_event_datas ring buffer
> > + * @irq_data_end: End index of irq_event_datas ring buffer
> > + * @irq_dequeue_lock: Lock to protect irq_event_datas ring buffer
> > + * @scp_mem_pa: DMA address for SCP device
> > + * @scp_mem_iova: DMA address for ISP HW DMA devices
> > + * @frame_seq_no: Sequence number for ISP frame buffer
> > + * @isp_hw_module: Active camera HW module
> > + * @num_clks: The number of driver's clock
> > + * @clk_list: The list of clock data
> > + * @lock: Lock to protect context operations
> > + *
> > + */
> > +struct mtk_isp_p1_ctx {
> > +	struct isp_queue composer_txlist;
> > +	struct isp_thread composer_tx_thread;
> > +	atomic_t cmd_queued;
> > +	atomic_t ipi_occupied;
> > +	atomic_t scp_state;
> > +	atomic_t composing_frame;
> > +	atomic_t composed_frame_id;
> > +	struct isp_thread composer_deinit_thread;
> > +	struct isp_queue p1_enqueue_list;
> > +	struct isp_thread isp_deque_thread;
> > +	struct mtk_cam_dev_stat_event_data irq_event_datas[IRQ_DATA_BUF_SIZE];
> > +	atomic_t irq_data_start;
> > +	atomic_t irq_data_end;
> > +	spinlock_t irq_dequeue_lock; /* ISP frame dequeuq protection */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +	dma_addr_t scp_mem_pa;
> > +	dma_addr_t scp_mem_iova;
> > +	int frame_seq_no;
> > +	unsigned int isp_hw_module;
> > +	unsigned int isp_raw_path;
> 
> Not documented above.
> 

Fix in next patch.

> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clk_list;
> > +	struct mutex lock; /* Protect context operations */
> 
> Already documented in kerneldoc.
> 

Fix in next patch.

> > +};
> > +
> > +struct isp_p1_device {
> > +	struct platform_device *pdev;
> > +	struct platform_device *scp_pdev;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_isp_p1_ctx isp_ctx;
> > +	struct mtk_cam_dev cam_dev;
> > +	struct isp_device isp_devs[ISP_DEV_NODE_NUM];
> > +};
> 
> Please document in a kerneldoc comment.
> 

Fix in next patch.

> > +
> > +static inline struct isp_p1_device *
> > +p1_ctx_to_dev(const struct mtk_isp_p1_ctx *__p1_ctx)
> > +{
> > +	return container_of(__p1_ctx, struct isp_p1_device, isp_ctx);
> > +}
> > +
> > +static inline struct isp_p1_device *get_p1_device(struct device *dev)
> > +{
> > +	return ((struct isp_p1_device *)dev_get_drvdata(dev));
> 
> No need to cast. And, I don't think we need a function for this, just call
> dev_get_drvdata() directly.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments .

Best regards,


Jungo




_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
  2019-07-10  9:58         ` Tomasz Figa
  (?)
@ 2019-07-21  2:18           ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-21  2:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Tomasz:

On Wed, 2019-07-10 at 18:58 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> > This patch adds communication with the co-processor on the SoC
> > through the SCP driver. It supports bi-directional commands
> > to exchange data and perform command flow control function.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
> >  3 files changed, 579 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thank you for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > new file mode 100644
> > index 000000000000..04519d0b942f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > @@ -0,0 +1,371 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/kthread.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +
> > +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> 
> Is there any point to set them if we are deinitalizing? Moreover,
> isp_composer_init() would set them once we start again.
> 

We will remove these variable assignments.

> > +
> > +	spin_lock(&isp_ctx->composer_txlist.lock);
> > +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> > +				 &isp_ctx->composer_txlist.queue,
> > +				 list_entry) {
> > +		list_del(&ipi_job->list_entry);
> > +		kfree(ipi_job);
> > +	}
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (isp_ctx->composer_tx_thread.thread) {
> > +		kthread_stop(isp_ctx->composer_tx_thread.thread);
> 
> Shouldn't the thread be stopped at this point already? If not, wouldn't the
> atomic_set() at the beginning of this function confuse it?
> 
> In any case, this should be greatly simplified after we move to a workqueue,
> with one work per one task to do, as per other comments.
> 

We will simplify the IPI sending mechanism and remove these kthread
handling.

> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		isp_ctx->composer_tx_thread.thread = NULL;
> > +	}
> > +
> > +	if (isp_ctx->composer_deinit_thread.thread) {
> > +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread = NULL;
> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> 
> No need to use the sync variant.
> 

We don't get this point. If we will call pm_runtime_get_sync in
mtk_isp_hw_init function, will we need to call
pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
As we know, we should call runtime pm functions in pair.

> > +}
> > +
> > +/*
> > + * Two kinds of flow control in isp_composer_tx_work.
> > + *
> > + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> > + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> > + * It is controlled by ipi_occupied.
> 
> ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?
> 

In this version, we use async. scp_ipi_send function call with wait = 0.
If kernel sends too many P1 IPI commands in short time, P1 task in SCP
may miss some IPI command due to the IPI command processing time and the
size of command queue in SCP side. In order to avoid this kind of
condition, we use ISP_COMPOSING_MAX_NUM to control the sending flow of
IPI command in kernel side. The ISP_COMPOSING_MAX_NUM is changed to 4
for Chromium EC OS. We just miss to update the comment here.

In new version, we will change to use sync. scp_ipi_send function call
with non-zero wait variable. Based on this, we could remove IPI command
flow control in P1 driver.

> > + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
> 
> What does it mean and why is it so?
> 

In the origin design, SCP_ISP_CMD & SCP_ISP_FRAME are sending in the
same command queue by order. However, if we receive ISP_CMD_DEINIT
command, we will like to send this command firstly to SCP before
SCP_ISP_FRAME are queued in the queue. So we need to have one command
prioritize design here. Btw, in the new design, SCP_ISP_CMD &
SCP_ISP_FRAME are sent independent and we can remove this. 

> > + *
> > + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> > + * It is controlled by composing_frame.
> > + * Frame buffer is sent by SCP_ISP_FRAME command.
> 
> Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
> and case 2?
> 

For case 2, it is related to frame request handling with CQ buffer.
We send frame request data via SCP_ISP_FRAME to compose CQ buffers in
SCP. The maximum CQ buffers in SCP are 3. So in kernel side, we can't
send any SCP_ISP_FRAME command to SCP when the CQ buffers are full in
SCP until ISP HW has output the new frame with the corresponding CQ
buffer.

In the new design, this will be controlled by mtk_cam_dev_req_try_queue
function with MTK_ISP_MAX_RUNNING_JOBS.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam->running_job_count);
	unsigned long flags;

	if (!cam->streaming || buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam->pending_job_lock);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam->pending_job_lock);

	list_for_each_entry_safe(req, req_prev, &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);

		mtk_isp_req_enqueue(cam, req);
	}
}

> > + */
> > +static int isp_composer_tx_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> > +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> > +	int ret;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible
> > +			(isp_ctx->composer_tx_thread.wq,
> > +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM &&
> > +			 atomic_read(&isp_ctx->composing_frame)
> > +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> > +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM) ||
> > +			 kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock(&composer_txlist->lock);
> > +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> > +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> > +						 &composer_txlist->queue,
> > +						 list_entry) {
> > +				if (isp_composer_work->type == SCP_ISP_CMD) {
> > +					dev_dbg(dev, "Found a cmd\n");
> > +					break;
> > +				}
> > +			}
> > +		} else {
> > +			if (atomic_read(&isp_ctx->composing_frame) >=
> > +				ISP_FRAME_COMPOSING_MAX_NUM) {
> > +				spin_unlock(&composer_txlist->lock);
> > +				continue;
> > +			}
> > +			isp_composer_work =
> > +			    list_first_entry_or_null
> > +				(&composer_txlist->queue,
> > +				 struct mtk_isp_queue_work,
> > +				 list_entry);
> > +		}
> 
> I don't understand why this special handling of CMD vs FRAME is here, so I
> might be missing something, but would we really lose anything if we just
> simply removed it and queued everything in order?
> 
> Moreover, in V4L2, buffer queue and control operations are serialized wrt
> each other, so we probably wouldn't even have a chance to hit a case when we
> need to prioritize a CMD IPI over a FRAME IPI.
> 

Yes, this implementation is complicated and we will remove
implementation in next patch. We will simplify current implementation by
using:
1. Use sync. scp_ipi_send function call
2. Use workqueue for SCP_ISP_FRAME sending

> > +
> > +		list_del(&isp_composer_work->list_entry);
> > +		atomic_dec(&composer_txlist->queue_cnt);
> > +		spin_unlock(&composer_txlist->lock);
> > +
> > +		if (isp_composer_work->type == SCP_ISP_CMD) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_CMD,
> > +				 &isp_composer_work->cmd,
> > +				 sizeof(isp_composer_work->cmd),
> > +				 0);
> > +			atomic_dec(&isp_ctx->cmd_queued);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			dev_dbg(dev,
> > +				"%s cmd id %d sent, %d ipi buf occupied",
> > +				__func__,
> > +				isp_composer_work->cmd.cmd_id,
> > +				atomic_read(&isp_ctx->ipi_occupied));
> > +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_FRAME,
> > +				 &isp_composer_work->frameparams,
> > +				 sizeof(isp_composer_work->frameparams),
> > +				 0);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			atomic_inc(&isp_ctx->composing_frame);
> 
> Why do we need composing frame here, if ipi_occupied already limits us to 3?
> 

If we send SCP_ISP_FRAME command, we need to increase ipi_occupied with
1 for IPI command sending command flow and increase composing_frame with
1 for CQ buffers composing. But this implementation will be removed.

> > +			dev_dbg(dev,
> > +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> > +				__func__,
> > +				isp_composer_work->frameparams.frame_seq_no,
> > +				atomic_read(&isp_ctx->ipi_occupied),
> > +				atomic_read(&isp_ctx->composing_frame));
> > +		} else {
> > +			dev_err(dev,
> > +				"ignore IPI type: %d!\n",
> > +				isp_composer_work->type);
> > +		}
> > +		kfree(isp_composer_work);
> > +	}
> > +	return ret;
> > +}
> 
> The function above is way too complicated than it should be. I'd suggest a
> model similar to what we ended up in the DIP driver:
> >  - a freezable workqueue created for ISP composing works,
> >  - each ISP composing work entry would have a struct work_struct embedded,
> >  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
> >    above,
> >  - the workqueue would keep a queue of works itself, so driver's own list
> >    wouldn't be needed anymore,
> >  - similarly, each execution of the work func would operate on its own ISP
> >    composing work, so things like checking for list emptiness, waiting for
> >    work to be queued, etc. wouldn't be needed,
> >  - freezability of the workqueue would ensure nice synchonization with
> >    system suspend/resume (although one would still need to wait for the
> >    hardware/firmware to complete).
> 
> WDYT?
> 

yes, we will adopt your suggestion to re-factor current implementation.
Below is new implementation.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

static void isp_tx_frame_worker(struct work_struct *work)
{
	struct mtk_cam_dev_request *req =
		container_of(work, struct mtk_cam_dev_request, frame_work);
	struct mtk_cam_dev *cam =
		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
}

> > +
> > +static int isp_composer_deinit_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> > +				 kthread_should_stop());
> > +
> > +	dev_dbg(dev, "%s run deinit", __func__);
> > +	isp_composer_deinit(isp_ctx);
> > +
> > +	return 0;
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> 
> Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> contain some extra bytes at the end.)
> 

The len parameter is the actual sending bytes from SCP to kernel.
In the runtime, it is only 6 bytes for isp_ack_info command
However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
union structure.

> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > +		return;
> > +
> > +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > +		dev_dbg(dev, "ack frame_num:%d",
> > +			ipi_msg->ack_info.frame_seq_no);
> > +		atomic_set(&isp_ctx->composed_frame_id,
> > +			   ipi_msg->ack_info.frame_seq_no);
> 
> I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> frames and we should handle that somehow.
> 

No, we use isp_ctx->composed_frame_id to save which frame sequence
number are composed done in SCP. In new design, we will move this from
isp_ctx to p1_dev.

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}

> > +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> > +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> > +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> > +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> > +	}
> > +
> > +	atomic_dec_return(&isp_ctx->ipi_occupied);
> > +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +}
> > +
> > +int isp_composer_init(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp_pdev,
> > +			       SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler,
> > +			       isp_ctx);
> > +	if (ret)
> > +		return ret;
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> > +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (!isp_ctx->composer_tx_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> > +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> > +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +		isp_ctx->composer_tx_thread.thread =
> > +			kthread_run(isp_composer_tx_work, isp_ctx,
> > +				    "isp_composer_tx");
> > +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_tx_thread.thread = NULL;
> > +			goto nomem;
> 
> Why nomem?
> 

It is wrong. Need to correct with
ERR_PTR(isp_ctx->composer_tx_thread.thread).
These kthread handling will be removed in next patch.

> > +		}
> > +	} else {
> > +		dev_warn(dev, "old tx thread is existed\n");
> 
> This shouldn't be possible to happen.
> 

Yes, it should not be happen. Otherwise, there is a bug.

> > +	}
> > +
> > +	if (!isp_ctx->composer_deinit_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread =
> > +			kthread_run(isp_composer_deinit_work, isp_ctx,
> > +				    "isp_composer_deinit_work");
> 
> Why do we need to deinit from another kthread?
> 

This code will be removed in next patch.

> > +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_deinit_thread.thread = NULL;
> > +			goto nomem;
> > +		}
> > +	} else {
> > +		dev_warn(dev, "old rx thread is existed\n");
> 
> rx? The code above seems to refer to deinit.
> 

Got it.

> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return 0;
> > +
> > +nomem:
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return -ENOMEM;
> 
> We should return the original error code here.
> 

Got it.

> > +}
> > +
> > +void isp_composer_enqueue(struct device *dev,
> > +			  void *data,
> > +			  enum mtk_isp_scp_type type)
> > +{
> > +	struct mtk_isp_queue_work *isp_composer_work;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

Fix in next patch.

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
> 
> For most of the cases, it should be possible to preallocate this, e.g.
> >  - for FRAME, this could be inside the request struct,
> >  - for buffer queue it could be inside the buffer struct.
> 
> I'd suggest making the caller responsible for allocating if needed.
> 

Fix in next patch.

> > +	isp_composer_work->type = type;
> > +
> > +	switch (type) {
> > +	case SCP_ISP_CMD:
> > +		memcpy(&isp_composer_work->cmd, data,
> > +		       sizeof(isp_composer_work->cmd));
> > +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> > +			isp_composer_work->cmd.cmd_id);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		atomic_inc(&isp_ctx->cmd_queued);
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		break;
> > +	case SCP_ISP_FRAME:
> > +		memcpy(&isp_composer_work->frameparams, data,
> > +		       sizeof(isp_composer_work->frameparams));
> > +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> > +			isp_composer_work->frameparams.frame_seq_no);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> 
> The code in both cases is almost exactly the same. The only difference is
> the memcpy destination and size and whether isp_ctx->cmd_queued is
> incremented or not.
> 
> The memcpy will go away if my comment above is addressed and so that would
> go down to making the cmd_queued increment conditional.
> 

This function will be removed in next patch.
We will call scp_ipi_send directly in the caller, such as:

void mtk_isp_hw_config(struct mtk_cam_dev *cam,
		       struct p1_config_param *config_param)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
	memcpy(&composer_tx_cmd.config_param, config_param,
	       sizeof(*config_param));

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
	composer_tx_cmd.is_stream_on = on;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);

	isp_composer_uninit(p1_dev);
}


> > +		break;
> > +	default:
> > +		break;
> > +	}
> > +}
> > +
> > +void isp_composer_hw_init(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> 
> Should we also specify the size of the buffer? Otherwise we could end up
> with some undetectable overruns.
> 

The size of SCP composer's memory is fixed to 0x200000.
Is it necessary to specify the size of this buffer?

#define MTK_ISP_COMPOSER_MEM_SIZE 0x200000

ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
			MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);

> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_meta_config(struct device *dev,
> > +			      unsigned int dma)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> > +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_config(struct device *dev,
> > +			    struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_stream(struct device *dev, int on)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_deinit(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +
> > +	/* Wait for ISP_CMD_DEINIT command is handled done */
> > +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> > +				 msecs_to_jiffies(2000));
> > +	if (ret)
> > +		return;
> > +
> > +	dev_warn(dev, "Timeout & local de-init\n");
> > +	isp_composer_deinit(isp_ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > new file mode 100644
> > index 000000000000..fbd8593e9c2d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > @@ -0,0 +1,207 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _MTK_ISP_SCP_H
> > +#define _MTK_ISP_SCP_H
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +/*
> > + * struct img_size - image size information.
> > + *
> > + * @w: image width, the unit is pixel
> > + * @h: image height, the unit is pixel
> > + * @xsize: bytes per line based on width.
> > + * @stride: bytes per line when changing line.
> > + *          Normally, calculate new STRIDE based on
> > + *          xsize + HW constrain(page or align).
> > + *
> > + */
> > +struct img_size {
> > +	__u32 w;
> > +	__u32 h;
> > +	__u32 xsize;
> > +	__u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct img_buffer - buffer address information.
> > + *
> > + * @iova: DMA address for external devices.
> > + * @scp_addr: SCP address for external co-process unit.
> > + *
> > + */
> > +struct img_buffer {
> > +	__u32 iova;
> > +	__u32 scp_addr;
> > +} __packed;
> > +
> > +struct p1_img_crop {
> > +	__u32 left;
> > +	__u32 top;
> > +	__u32 width;
> > +	__u32 height;
> > +} __packed;
> > +
> > +struct p1_img_output {
> > +	struct img_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	__u8 pixel_byte;
> > +	__u32 img_fmt;
> > +} __packed;
> 
> Please document.
> 

Fix in next patch.

> > +
> > +/*
> > + * struct cfg_in_param - image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: indicate the sensor mode.
> > + *              1: continuous
> > + *              0: single
> > + * @subsample: indicate to enables SOF subsample or not.
> > + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: describe input data pattern.
> > + * @raw_pixel_id: bayer sequence.
> > + * @tg_fps: the fps rate of TG (time generator).
> > + * @img_fmt: the image format of input source.
> > + * @p1_img_crop: the crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	__u8 continuous;
> > +	__u8 subsample;
> > +	__u8 pixel_mode;
> > +	__u8 data_pattern;
> > +	__u8 raw_pixel_id;
> > +	__u16 tg_fps;
> > +	__u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - the image output parameters of main stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @pure_raw: indicate the image path control.
> > + *            1: pure raw
> > + *            0: processing raw
> > + * @pure_raw_pack: indicate the image is packed or not.
> > + *                 1: packed mode
> > + *                 0: unpacked mode
> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	/* Bypass main out parameters */
> > +	__u8 bypass;
> > +	/* Control HW image raw path */
> > +	__u8 pure_raw;
> > +	/* Control HW image pack function */
> 
> No need for these inline comments.
> 

Fix in next patch.

> > +	__u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - the image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	/* Bypass resize parameters */
> 
> No need for this inline comment.
> 

Fix in next patch.

> > +	__u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_meta_out_param - output meta information.
> > + *
> > + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> > + *
> > + */
> > +struct cfg_meta_out_param {
> > +	__u32 enabled_meta_dmas;
> > +} __packed;
> > +
> > +struct p1_config_param {
> > +	/* Sensor/TG info */
> > +	struct cfg_in_param cfg_in_param;
> > +	/* IMGO DMA */
> > +	struct cfg_main_out_param cfg_main_param;
> > +	/* RRZO DMA */
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	/* 3A DMAs and other. */
> > +	struct cfg_meta_out_param cfg_meta_param;
> 
> Please change the inline comments to a kerneldoc comment at the top.
> 

Fix in next patch.

> > +} __packed;
> > +
> > +struct p1_frame_param {
> > +	/* frame sequence number */
> > +	__u32 frame_seq_no;
> > +	/* SOF index */
> > +	__u32 sof_idx;
> > +	/* The memory address of tuning buffer from user space */
> 
> Ditto.
> 

Fix in next patch.

> > +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +struct P1_meta_frame {
> > +	__u32 enabled_dma;
> > +	__u32 vb_index;
> > +	struct img_buffer meta_addr;
> > +} __packed;
> > +
> > +struct isp_init_info {
> > +	__u8 hw_module;
> > +	struct img_buffer cq_addr;
> > +} __packed;
> > +
> > +struct isp_ack_info {
> > +	__u8 cmd_id;
> > +	__u32 frame_seq_no;
> > +} __packed;
> > +
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_CONFIG_META,
> > +	ISP_CMD_ENQUEUE_META,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +struct mtk_isp_scp_p1_cmd {
> > +	__u8 cmd_id;
> > +	union {
> > +		struct isp_init_info frameparam;
> > +		struct p1_config_param config_param;
> > +		struct cfg_meta_out_param cfg_meta_out_param;
> > +		struct P1_meta_frame meta_frame;
> > +		__u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +enum mtk_isp_scp_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> 
> Please document all the structs and enum above using kerneldoc.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments.

Best regards,


Jungo 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-21  2:18           ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-21  2:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: hverkuil, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, sean.cheng, sj.huang, frederic.chen, ryan.yu,
	rynn.wu, frankie.chiu

Hi Tomasz:

On Wed, 2019-07-10 at 18:58 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> > This patch adds communication with the co-processor on the SoC
> > through the SCP driver. It supports bi-directional commands
> > to exchange data and perform command flow control function.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
> >  3 files changed, 579 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thank you for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > new file mode 100644
> > index 000000000000..04519d0b942f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > @@ -0,0 +1,371 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/kthread.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +
> > +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> 
> Is there any point to set them if we are deinitalizing? Moreover,
> isp_composer_init() would set them once we start again.
> 

We will remove these variable assignments.

> > +
> > +	spin_lock(&isp_ctx->composer_txlist.lock);
> > +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> > +				 &isp_ctx->composer_txlist.queue,
> > +				 list_entry) {
> > +		list_del(&ipi_job->list_entry);
> > +		kfree(ipi_job);
> > +	}
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (isp_ctx->composer_tx_thread.thread) {
> > +		kthread_stop(isp_ctx->composer_tx_thread.thread);
> 
> Shouldn't the thread be stopped at this point already? If not, wouldn't the
> atomic_set() at the beginning of this function confuse it?
> 
> In any case, this should be greatly simplified after we move to a workqueue,
> with one work per one task to do, as per other comments.
> 

We will simplify the IPI sending mechanism and remove these kthread
handling.

> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		isp_ctx->composer_tx_thread.thread = NULL;
> > +	}
> > +
> > +	if (isp_ctx->composer_deinit_thread.thread) {
> > +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread = NULL;
> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> 
> No need to use the sync variant.
> 

We don't get this point. If we will call pm_runtime_get_sync in
mtk_isp_hw_init function, will we need to call
pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
As we know, we should call runtime pm functions in pair.

> > +}
> > +
> > +/*
> > + * Two kinds of flow control in isp_composer_tx_work.
> > + *
> > + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> > + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> > + * It is controlled by ipi_occupied.
> 
> ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?
> 

In this version, we use async. scp_ipi_send function call with wait = 0.
If kernel sends too many P1 IPI commands in short time, P1 task in SCP
may miss some IPI command due to the IPI command processing time and the
size of command queue in SCP side. In order to avoid this kind of
condition, we use ISP_COMPOSING_MAX_NUM to control the sending flow of
IPI command in kernel side. The ISP_COMPOSING_MAX_NUM is changed to 4
for Chromium EC OS. We just miss to update the comment here.

In new version, we will change to use sync. scp_ipi_send function call
with non-zero wait variable. Based on this, we could remove IPI command
flow control in P1 driver.

> > + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
> 
> What does it mean and why is it so?
> 

In the origin design, SCP_ISP_CMD & SCP_ISP_FRAME are sending in the
same command queue by order. However, if we receive ISP_CMD_DEINIT
command, we will like to send this command firstly to SCP before
SCP_ISP_FRAME are queued in the queue. So we need to have one command
prioritize design here. Btw, in the new design, SCP_ISP_CMD &
SCP_ISP_FRAME are sent independent and we can remove this. 

> > + *
> > + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> > + * It is controlled by composing_frame.
> > + * Frame buffer is sent by SCP_ISP_FRAME command.
> 
> Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
> and case 2?
> 

For case 2, it is related to frame request handling with CQ buffer.
We send frame request data via SCP_ISP_FRAME to compose CQ buffers in
SCP. The maximum CQ buffers in SCP are 3. So in kernel side, we can't
send any SCP_ISP_FRAME command to SCP when the CQ buffers are full in
SCP until ISP HW has output the new frame with the corresponding CQ
buffer.

In the new design, this will be controlled by mtk_cam_dev_req_try_queue
function with MTK_ISP_MAX_RUNNING_JOBS.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam->running_job_count);
	unsigned long flags;

	if (!cam->streaming || buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam->pending_job_lock);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam->pending_job_lock);

	list_for_each_entry_safe(req, req_prev, &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);

		mtk_isp_req_enqueue(cam, req);
	}
}

> > + */
> > +static int isp_composer_tx_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> > +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> > +	int ret;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible
> > +			(isp_ctx->composer_tx_thread.wq,
> > +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM &&
> > +			 atomic_read(&isp_ctx->composing_frame)
> > +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> > +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM) ||
> > +			 kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock(&composer_txlist->lock);
> > +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> > +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> > +						 &composer_txlist->queue,
> > +						 list_entry) {
> > +				if (isp_composer_work->type == SCP_ISP_CMD) {
> > +					dev_dbg(dev, "Found a cmd\n");
> > +					break;
> > +				}
> > +			}
> > +		} else {
> > +			if (atomic_read(&isp_ctx->composing_frame) >=
> > +				ISP_FRAME_COMPOSING_MAX_NUM) {
> > +				spin_unlock(&composer_txlist->lock);
> > +				continue;
> > +			}
> > +			isp_composer_work =
> > +			    list_first_entry_or_null
> > +				(&composer_txlist->queue,
> > +				 struct mtk_isp_queue_work,
> > +				 list_entry);
> > +		}
> 
> I don't understand why this special handling of CMD vs FRAME is here, so I
> might be missing something, but would we really lose anything if we just
> simply removed it and queued everything in order?
> 
> Moreover, in V4L2, buffer queue and control operations are serialized wrt
> each other, so we probably wouldn't even have a chance to hit a case when we
> need to prioritize a CMD IPI over a FRAME IPI.
> 

Yes, this implementation is complicated and we will remove
implementation in next patch. We will simplify current implementation by
using:
1. Use sync. scp_ipi_send function call
2. Use workqueue for SCP_ISP_FRAME sending

> > +
> > +		list_del(&isp_composer_work->list_entry);
> > +		atomic_dec(&composer_txlist->queue_cnt);
> > +		spin_unlock(&composer_txlist->lock);
> > +
> > +		if (isp_composer_work->type == SCP_ISP_CMD) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_CMD,
> > +				 &isp_composer_work->cmd,
> > +				 sizeof(isp_composer_work->cmd),
> > +				 0);
> > +			atomic_dec(&isp_ctx->cmd_queued);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			dev_dbg(dev,
> > +				"%s cmd id %d sent, %d ipi buf occupied",
> > +				__func__,
> > +				isp_composer_work->cmd.cmd_id,
> > +				atomic_read(&isp_ctx->ipi_occupied));
> > +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_FRAME,
> > +				 &isp_composer_work->frameparams,
> > +				 sizeof(isp_composer_work->frameparams),
> > +				 0);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			atomic_inc(&isp_ctx->composing_frame);
> 
> Why do we need composing frame here, if ipi_occupied already limits us to 3?
> 

If we send SCP_ISP_FRAME command, we need to increase ipi_occupied with
1 for IPI command sending command flow and increase composing_frame with
1 for CQ buffers composing. But this implementation will be removed.

> > +			dev_dbg(dev,
> > +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> > +				__func__,
> > +				isp_composer_work->frameparams.frame_seq_no,
> > +				atomic_read(&isp_ctx->ipi_occupied),
> > +				atomic_read(&isp_ctx->composing_frame));
> > +		} else {
> > +			dev_err(dev,
> > +				"ignore IPI type: %d!\n",
> > +				isp_composer_work->type);
> > +		}
> > +		kfree(isp_composer_work);
> > +	}
> > +	return ret;
> > +}
> 
> The function above is way too complicated than it should be. I'd suggest a
> model similar to what we ended up in the DIP driver:
> >  - a freezable workqueue created for ISP composing works,
> >  - each ISP composing work entry would have a struct work_struct embedded,
> >  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
> >    above,
> >  - the workqueue would keep a queue of works itself, so driver's own list
> >    wouldn't be needed anymore,
> >  - similarly, each execution of the work func would operate on its own ISP
> >    composing work, so things like checking for list emptiness, waiting for
> >    work to be queued, etc. wouldn't be needed,
> >  - freezability of the workqueue would ensure nice synchonization with
> >    system suspend/resume (although one would still need to wait for the
> >    hardware/firmware to complete).
> 
> WDYT?
> 

yes, we will adopt your suggestion to re-factor current implementation.
Below is new implementation.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

static void isp_tx_frame_worker(struct work_struct *work)
{
	struct mtk_cam_dev_request *req =
		container_of(work, struct mtk_cam_dev_request, frame_work);
	struct mtk_cam_dev *cam =
		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
}

> > +
> > +static int isp_composer_deinit_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> > +				 kthread_should_stop());
> > +
> > +	dev_dbg(dev, "%s run deinit", __func__);
> > +	isp_composer_deinit(isp_ctx);
> > +
> > +	return 0;
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> 
> Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> contain some extra bytes at the end.)
> 

The len parameter is the actual sending bytes from SCP to kernel.
In the runtime, it is only 6 bytes for isp_ack_info command
However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
union structure.

> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > +		return;
> > +
> > +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > +		dev_dbg(dev, "ack frame_num:%d",
> > +			ipi_msg->ack_info.frame_seq_no);
> > +		atomic_set(&isp_ctx->composed_frame_id,
> > +			   ipi_msg->ack_info.frame_seq_no);
> 
> I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> frames and we should handle that somehow.
> 

No, we use isp_ctx->composed_frame_id to save which frame sequence
number are composed done in SCP. In new design, we will move this from
isp_ctx to p1_dev.

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}

> > +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> > +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> > +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> > +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> > +	}
> > +
> > +	atomic_dec_return(&isp_ctx->ipi_occupied);
> > +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +}
> > +
> > +int isp_composer_init(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp_pdev,
> > +			       SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler,
> > +			       isp_ctx);
> > +	if (ret)
> > +		return ret;
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> > +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (!isp_ctx->composer_tx_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> > +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> > +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +		isp_ctx->composer_tx_thread.thread =
> > +			kthread_run(isp_composer_tx_work, isp_ctx,
> > +				    "isp_composer_tx");
> > +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_tx_thread.thread = NULL;
> > +			goto nomem;
> 
> Why nomem?
> 

It is wrong. Need to correct with
ERR_PTR(isp_ctx->composer_tx_thread.thread).
These kthread handling will be removed in next patch.

> > +		}
> > +	} else {
> > +		dev_warn(dev, "old tx thread is existed\n");
> 
> This shouldn't be possible to happen.
> 

Yes, it should not be happen. Otherwise, there is a bug.

> > +	}
> > +
> > +	if (!isp_ctx->composer_deinit_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread =
> > +			kthread_run(isp_composer_deinit_work, isp_ctx,
> > +				    "isp_composer_deinit_work");
> 
> Why do we need to deinit from another kthread?
> 

This code will be removed in next patch.

> > +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_deinit_thread.thread = NULL;
> > +			goto nomem;
> > +		}
> > +	} else {
> > +		dev_warn(dev, "old rx thread is existed\n");
> 
> rx? The code above seems to refer to deinit.
> 

Got it.

> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return 0;
> > +
> > +nomem:
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return -ENOMEM;
> 
> We should return the original error code here.
> 

Got it.

> > +}
> > +
> > +void isp_composer_enqueue(struct device *dev,
> > +			  void *data,
> > +			  enum mtk_isp_scp_type type)
> > +{
> > +	struct mtk_isp_queue_work *isp_composer_work;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

Fix in next patch.

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
> 
> For most of the cases, it should be possible to preallocate this, e.g.
> >  - for FRAME, this could be inside the request struct,
> >  - for buffer queue it could be inside the buffer struct.
> 
> I'd suggest making the caller responsible for allocating if needed.
> 

Fix in next patch.

> > +	isp_composer_work->type = type;
> > +
> > +	switch (type) {
> > +	case SCP_ISP_CMD:
> > +		memcpy(&isp_composer_work->cmd, data,
> > +		       sizeof(isp_composer_work->cmd));
> > +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> > +			isp_composer_work->cmd.cmd_id);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		atomic_inc(&isp_ctx->cmd_queued);
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		break;
> > +	case SCP_ISP_FRAME:
> > +		memcpy(&isp_composer_work->frameparams, data,
> > +		       sizeof(isp_composer_work->frameparams));
> > +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> > +			isp_composer_work->frameparams.frame_seq_no);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> 
> The code in both cases is almost exactly the same. The only difference is
> the memcpy destination and size and whether isp_ctx->cmd_queued is
> incremented or not.
> 
> The memcpy will go away if my comment above is addressed and so that would
> go down to making the cmd_queued increment conditional.
> 

This function will be removed in next patch.
We will call scp_ipi_send directly in the caller, such as:

void mtk_isp_hw_config(struct mtk_cam_dev *cam,
		       struct p1_config_param *config_param)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
	memcpy(&composer_tx_cmd.config_param, config_param,
	       sizeof(*config_param));

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
	composer_tx_cmd.is_stream_on = on;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);

	isp_composer_uninit(p1_dev);
}


> > +		break;
> > +	default:
> > +		break;
> > +	}
> > +}
> > +
> > +void isp_composer_hw_init(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> 
> Should we also specify the size of the buffer? Otherwise we could end up
> with some undetectable overruns.
> 

The size of SCP composer's memory is fixed to 0x200000.
Is it necessary to specify the size of this buffer?

#define MTK_ISP_COMPOSER_MEM_SIZE 0x200000

ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
			MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);

> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_meta_config(struct device *dev,
> > +			      unsigned int dma)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> > +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_config(struct device *dev,
> > +			    struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_stream(struct device *dev, int on)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_deinit(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +
> > +	/* Wait for ISP_CMD_DEINIT command is handled done */
> > +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> > +				 msecs_to_jiffies(2000));
> > +	if (ret)
> > +		return;
> > +
> > +	dev_warn(dev, "Timeout & local de-init\n");
> > +	isp_composer_deinit(isp_ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > new file mode 100644
> > index 000000000000..fbd8593e9c2d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > @@ -0,0 +1,207 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _MTK_ISP_SCP_H
> > +#define _MTK_ISP_SCP_H
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +/*
> > + * struct img_size - image size information.
> > + *
> > + * @w: image width, the unit is pixel
> > + * @h: image height, the unit is pixel
> > + * @xsize: bytes per line based on width.
> > + * @stride: bytes per line when changing line.
> > + *          Normally, calculate new STRIDE based on
> > + *          xsize + HW constrain(page or align).
> > + *
> > + */
> > +struct img_size {
> > +	__u32 w;
> > +	__u32 h;
> > +	__u32 xsize;
> > +	__u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct img_buffer - buffer address information.
> > + *
> > + * @iova: DMA address for external devices.
> > + * @scp_addr: SCP address for external co-process unit.
> > + *
> > + */
> > +struct img_buffer {
> > +	__u32 iova;
> > +	__u32 scp_addr;
> > +} __packed;
> > +
> > +struct p1_img_crop {
> > +	__u32 left;
> > +	__u32 top;
> > +	__u32 width;
> > +	__u32 height;
> > +} __packed;
> > +
> > +struct p1_img_output {
> > +	struct img_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	__u8 pixel_byte;
> > +	__u32 img_fmt;
> > +} __packed;
> 
> Please document.
> 

Fix in next patch.

> > +
> > +/*
> > + * struct cfg_in_param - image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: indicate the sensor mode.
> > + *              1: continuous
> > + *              0: single
> > + * @subsample: indicate to enables SOF subsample or not.
> > + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: describe input data pattern.
> > + * @raw_pixel_id: bayer sequence.
> > + * @tg_fps: the fps rate of TG (time generator).
> > + * @img_fmt: the image format of input source.
> > + * @p1_img_crop: the crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	__u8 continuous;
> > +	__u8 subsample;
> > +	__u8 pixel_mode;
> > +	__u8 data_pattern;
> > +	__u8 raw_pixel_id;
> > +	__u16 tg_fps;
> > +	__u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - the image output parameters of main stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @pure_raw: indicate the image path control.
> > + *            1: pure raw
> > + *            0: processing raw
> > + * @pure_raw_pack: indicate the image is packed or not.
> > + *                 1: packed mode
> > + *                 0: unpacked mode
> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	/* Bypass main out parameters */
> > +	__u8 bypass;
> > +	/* Control HW image raw path */
> > +	__u8 pure_raw;
> > +	/* Control HW image pack function */
> 
> No need for these inline comments.
> 

Fix in next patch.

> > +	__u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - the image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	/* Bypass resize parameters */
> 
> No need for this inline comment.
> 

Fix in next patch.

> > +	__u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_meta_out_param - output meta information.
> > + *
> > + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> > + *
> > + */
> > +struct cfg_meta_out_param {
> > +	__u32 enabled_meta_dmas;
> > +} __packed;
> > +
> > +struct p1_config_param {
> > +	/* Sensor/TG info */
> > +	struct cfg_in_param cfg_in_param;
> > +	/* IMGO DMA */
> > +	struct cfg_main_out_param cfg_main_param;
> > +	/* RRZO DMA */
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	/* 3A DMAs and other. */
> > +	struct cfg_meta_out_param cfg_meta_param;
> 
> Please change the inline comments to a kerneldoc comment at the top.
> 

Fix in next patch.

> > +} __packed;
> > +
> > +struct p1_frame_param {
> > +	/* frame sequence number */
> > +	__u32 frame_seq_no;
> > +	/* SOF index */
> > +	__u32 sof_idx;
> > +	/* The memory address of tuning buffer from user space */
> 
> Ditto.
> 

Fix in next patch.

> > +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +struct P1_meta_frame {
> > +	__u32 enabled_dma;
> > +	__u32 vb_index;
> > +	struct img_buffer meta_addr;
> > +} __packed;
> > +
> > +struct isp_init_info {
> > +	__u8 hw_module;
> > +	struct img_buffer cq_addr;
> > +} __packed;
> > +
> > +struct isp_ack_info {
> > +	__u8 cmd_id;
> > +	__u32 frame_seq_no;
> > +} __packed;
> > +
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_CONFIG_META,
> > +	ISP_CMD_ENQUEUE_META,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +struct mtk_isp_scp_p1_cmd {
> > +	__u8 cmd_id;
> > +	union {
> > +		struct isp_init_info frameparam;
> > +		struct p1_config_param config_param;
> > +		struct cfg_meta_out_param cfg_meta_out_param;
> > +		struct P1_meta_frame meta_frame;
> > +		__u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +enum mtk_isp_scp_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> 
> Please document all the structs and enum above using kerneldoc.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments.

Best regards,


Jungo 



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-21  2:18           ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-21  2:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, sean.cheng, frederic.chen, rynn.wu, srv_heupstream,
	robh, ryan.yu, frankie.chiu, hverkuil, ddavenport, sj.huang,
	linux-mediatek, laurent.pinchart, matthias.bgg, mchehab,
	linux-arm-kernel, linux-media

Hi Tomasz:

On Wed, 2019-07-10 at 18:58 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Tue, Jun 11, 2019 at 11:53:43AM +0800, Jungo Lin wrote:
> > This patch adds communication with the co-processor on the SoC
> > through the SCP driver. It supports bi-directional commands
> > to exchange data and perform command flow control function.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > This patch depends on "Add support for mt8183 SCP"[1].
> > 
> > [1] https://patchwork.kernel.org/cover/10972143/
> > ---
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |   1 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.c | 371 ++++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-scp.h | 207 ++++++++++
> >  3 files changed, 579 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > 
> 
> Thanks for the patch! Please see my comments inline.
> 
> [snip]
> 

Thank you for your comments. Please check my replies inline.

[snip]

> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > new file mode 100644
> > index 000000000000..04519d0b942f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.c
> > @@ -0,0 +1,371 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2018 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/kthread.h>
> > +#include <linux/platform_data/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include "mtk_cam.h"
> > +
> > +static void isp_composer_deinit(struct mtk_isp_p1_ctx *isp_ctx)
> > +{
> > +	struct mtk_isp_queue_work *ipi_job, *tmp_ipi_job;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> 
> Is there any point to set them if we are deinitalizing? Moreover,
> isp_composer_init() would set them once we start again.
> 

We will remove these variable assignments.

> > +
> > +	spin_lock(&isp_ctx->composer_txlist.lock);
> > +	list_for_each_entry_safe(ipi_job, tmp_ipi_job,
> > +				 &isp_ctx->composer_txlist.queue,
> > +				 list_entry) {
> > +		list_del(&ipi_job->list_entry);
> > +		kfree(ipi_job);
> > +	}
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (isp_ctx->composer_tx_thread.thread) {
> > +		kthread_stop(isp_ctx->composer_tx_thread.thread);
> 
> Shouldn't the thread be stopped at this point already? If not, wouldn't the
> atomic_set() at the beginning of this function confuse it?
> 
> In any case, this should be greatly simplified after we move to a workqueue,
> with one work per one task to do, as per other comments.
> 

We will simplify the IPI sending mechanism and remove these kthread
handling.

> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		isp_ctx->composer_tx_thread.thread = NULL;
> > +	}
> > +
> > +	if (isp_ctx->composer_deinit_thread.thread) {
> > +		wake_up(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread = NULL;
> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	pm_runtime_put_sync(&p1_dev->pdev->dev);
> 
> No need to use the sync variant.
> 

We don't get this point. If we will call pm_runtime_get_sync in
mtk_isp_hw_init function, will we need to call
pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
As we know, we should call runtime pm functions in pair.

> > +}
> > +
> > +/*
> > + * Two kinds of flow control in isp_composer_tx_work.
> > + *
> > + * Case 1: IPI commands flow control. The maximum number of command queues is 3.
> > + * There are two types of IPI commands (SCP_ISP_CMD/SCP_ISP_FRAME) in P1 driver.
> > + * It is controlled by ipi_occupied.
> 
> ISP_COMPOSING_MAX_NUM is defined to 4, not 3. Is that expected?
> 

In this version, we use async. scp_ipi_send function call with wait = 0.
If kernel sends too many P1 IPI commands in short time, P1 task in SCP
may miss some IPI command due to the IPI command processing time and the
size of command queue in SCP side. In order to avoid this kind of
condition, we use ISP_COMPOSING_MAX_NUM to control the sending flow of
IPI command in kernel side. The ISP_COMPOSING_MAX_NUM is changed to 4
for Chromium EC OS. We just miss to update the comment here.

In new version, we will change to use sync. scp_ipi_send function call
with non-zero wait variable. Based on this, we could remove IPI command
flow control in P1 driver.

> > + * The priority of SCP_ISP_CMD is higher than SCP_ISP_FRAME.
> 
> What does it mean and why is it so?
> 

In the origin design, SCP_ISP_CMD & SCP_ISP_FRAME are sending in the
same command queue by order. However, if we receive ISP_CMD_DEINIT
command, we will like to send this command firstly to SCP before
SCP_ISP_FRAME are queued in the queue. So we need to have one command
prioritize design here. Btw, in the new design, SCP_ISP_CMD &
SCP_ISP_FRAME are sent independent and we can remove this. 

> > + *
> > + * Case 2: Frame buffers flow control. The maximum number of frame buffers is 3.
> > + * It is controlled by composing_frame.
> > + * Frame buffer is sent by SCP_ISP_FRAME command.
> 
> Case 1 already mentions SCP_ISP_FRAME. What's the difference between that
> and case 2?
> 

For case 2, it is related to frame request handling with CQ buffer.
We send frame request data via SCP_ISP_FRAME to compose CQ buffers in
SCP. The maximum CQ buffers in SCP are 3. So in kernel side, we can't
send any SCP_ISP_FRAME command to SCP when the CQ buffers are full in
SCP until ISP HW has output the new frame with the corresponding CQ
buffer.

In the new design, this will be controlled by mtk_cam_dev_req_try_queue
function with MTK_ISP_MAX_RUNNING_JOBS.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	struct list_head enqueue_job_list;
	int buffer_cnt = atomic_read(&cam->running_job_count);
	unsigned long flags;

	if (!cam->streaming || buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "stream off or buffers are full:%d\n",
			buffer_cnt);
		return;
	}

	INIT_LIST_HEAD(&enqueue_job_list);

	spin_lock(&cam->pending_job_lock);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		list_add_tail(&req->list, &enqueue_job_list);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock(&cam->pending_job_lock);

	list_for_each_entry_safe(req, req_prev, &enqueue_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);

		mtk_isp_req_enqueue(cam, req);
	}
}

> > + */
> > +static int isp_composer_tx_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_queue_work *isp_composer_work, *tmp_ipi_job;
> > +	struct isp_queue *composer_txlist = &isp_ctx->composer_txlist;
> > +	int ret;
> > +
> > +	while (1) {
> > +		ret = wait_event_interruptible
> > +			(isp_ctx->composer_tx_thread.wq,
> > +			 (atomic_read(&composer_txlist->queue_cnt) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM &&
> > +			 atomic_read(&isp_ctx->composing_frame)
> > +				< ISP_FRAME_COMPOSING_MAX_NUM) ||
> > +			 (atomic_read(&isp_ctx->cmd_queued) > 0 &&
> > +			 atomic_read(&isp_ctx->ipi_occupied)
> > +				< ISP_COMPOSING_MAX_NUM) ||
> > +			 kthread_should_stop());
> > +
> > +		if (kthread_should_stop())
> > +			break;
> > +
> > +		spin_lock(&composer_txlist->lock);
> > +		if (atomic_read(&isp_ctx->cmd_queued) > 0) {
> > +			list_for_each_entry_safe(isp_composer_work, tmp_ipi_job,
> > +						 &composer_txlist->queue,
> > +						 list_entry) {
> > +				if (isp_composer_work->type == SCP_ISP_CMD) {
> > +					dev_dbg(dev, "Found a cmd\n");
> > +					break;
> > +				}
> > +			}
> > +		} else {
> > +			if (atomic_read(&isp_ctx->composing_frame) >=
> > +				ISP_FRAME_COMPOSING_MAX_NUM) {
> > +				spin_unlock(&composer_txlist->lock);
> > +				continue;
> > +			}
> > +			isp_composer_work =
> > +			    list_first_entry_or_null
> > +				(&composer_txlist->queue,
> > +				 struct mtk_isp_queue_work,
> > +				 list_entry);
> > +		}
> 
> I don't understand why this special handling of CMD vs FRAME is here, so I
> might be missing something, but would we really lose anything if we just
> simply removed it and queued everything in order?
> 
> Moreover, in V4L2, buffer queue and control operations are serialized wrt
> each other, so we probably wouldn't even have a chance to hit a case when we
> need to prioritize a CMD IPI over a FRAME IPI.
> 

Yes, this implementation is complicated and we will remove
implementation in next patch. We will simplify current implementation by
using:
1. Use sync. scp_ipi_send function call
2. Use workqueue for SCP_ISP_FRAME sending

> > +
> > +		list_del(&isp_composer_work->list_entry);
> > +		atomic_dec(&composer_txlist->queue_cnt);
> > +		spin_unlock(&composer_txlist->lock);
> > +
> > +		if (isp_composer_work->type == SCP_ISP_CMD) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_CMD,
> > +				 &isp_composer_work->cmd,
> > +				 sizeof(isp_composer_work->cmd),
> > +				 0);
> > +			atomic_dec(&isp_ctx->cmd_queued);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			dev_dbg(dev,
> > +				"%s cmd id %d sent, %d ipi buf occupied",
> > +				__func__,
> > +				isp_composer_work->cmd.cmd_id,
> > +				atomic_read(&isp_ctx->ipi_occupied));
> > +		} else if (isp_composer_work->type == SCP_ISP_FRAME) {
> > +			scp_ipi_send
> > +				(p1_dev->scp_pdev,
> > +				 SCP_IPI_ISP_FRAME,
> > +				 &isp_composer_work->frameparams,
> > +				 sizeof(isp_composer_work->frameparams),
> > +				 0);
> > +			atomic_inc(&isp_ctx->ipi_occupied);
> > +			atomic_inc(&isp_ctx->composing_frame);
> 
> Why do we need composing frame here, if ipi_occupied already limits us to 3?
> 

If we send SCP_ISP_FRAME command, we need to increase ipi_occupied with
1 for IPI command sending command flow and increase composing_frame with
1 for CQ buffers composing. But this implementation will be removed.

> > +			dev_dbg(dev,
> > +				"%s frame %d sent, %d ipi, %d CQ bufs occupied",
> > +				__func__,
> > +				isp_composer_work->frameparams.frame_seq_no,
> > +				atomic_read(&isp_ctx->ipi_occupied),
> > +				atomic_read(&isp_ctx->composing_frame));
> > +		} else {
> > +			dev_err(dev,
> > +				"ignore IPI type: %d!\n",
> > +				isp_composer_work->type);
> > +		}
> > +		kfree(isp_composer_work);
> > +	}
> > +	return ret;
> > +}
> 
> The function above is way too complicated than it should be. I'd suggest a
> model similar to what we ended up in the DIP driver:
> >  - a freezable workqueue created for ISP composing works,
> >  - each ISP composing work entry would have a struct work_struct embedded,
> >  - isp_composer_enqueue() would enqueue the work_struct to the workqueue
> >    above,
> >  - the workqueue would keep a queue of works itself, so driver's own list
> >    wouldn't be needed anymore,
> >  - similarly, each execution of the work func would operate on its own ISP
> >    composing work, so things like checking for list emptiness, waiting for
> >    work to be queued, etc. wouldn't be needed,
> >  - freezability of the workqueue would ensure nice synchonization with
> >    system suspend/resume (although one would still need to wait for the
> >    hardware/firmware to complete).
> 
> WDYT?
> 

yes, we will adopt your suggestion to re-factor current implementation.
Below is new implementation.

void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
	int ret;

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
	ret = queue_work(p1_dev->composer_wq, &req->frame_work);
	if (!ret)
		dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
			req->frame_params.frame_seq_no, ret);
	else
		dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
			req->req.debug_str, req->frame_params.frame_seq_no,
			atomic_read(&cam->running_job_count));
}

static void isp_tx_frame_worker(struct work_struct *work)
{
	struct mtk_cam_dev_request *req =
		container_of(work, struct mtk_cam_dev_request, frame_work);
	struct mtk_cam_dev *cam =
		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
}

> > +
> > +static int isp_composer_deinit_work(void *data)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)data;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(data);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +
> > +	wait_event_interruptible(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF ||
> > +				 kthread_should_stop());
> > +
> > +	dev_dbg(dev, "%s run deinit", __func__);
> > +	isp_composer_deinit(isp_ctx);
> > +
> > +	return 0;
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > +	struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > +	struct device *dev = &p1_dev->pdev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> 
> Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> contain some extra bytes at the end.)
> 

The len parameter is the actual sending bytes from SCP to kernel.
In the runtime, it is only 6 bytes for isp_ack_info command
However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
union structure.

> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > +		return;
> > +
> > +	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > +		dev_dbg(dev, "ack frame_num:%d",
> > +			ipi_msg->ack_info.frame_seq_no);
> > +		atomic_set(&isp_ctx->composed_frame_id,
> > +			   ipi_msg->ack_info.frame_seq_no);
> 
> I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> frames and we should handle that somehow.
> 

No, we use isp_ctx->composed_frame_id to save which frame sequence
number are composed done in SCP. In new design, we will move this from
isp_ctx to p1_dev.

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}

> > +	} else if (ipi_msg->ack_info.cmd_id == ISP_CMD_DEINIT) {
> > +		dev_dbg(dev, "ISP_CMD_DEINIT is acked");
> > +		atomic_set(&isp_ctx->scp_state, SCP_OFF);
> > +		wake_up_interruptible(&isp_ctx->composer_deinit_thread.wq);
> > +	}
> > +
> > +	atomic_dec_return(&isp_ctx->ipi_occupied);
> > +	wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +}
> > +
> > +int isp_composer_init(struct device *dev)
> > +{
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp_pdev,
> > +			       SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler,
> > +			       isp_ctx);
> > +	if (ret)
> > +		return ret;
> > +
> > +	atomic_set(&isp_ctx->cmd_queued, 0);
> > +	atomic_set(&isp_ctx->composer_txlist.queue_cnt, 0);
> > +	atomic_set(&isp_ctx->composing_frame, 0);
> > +	atomic_set(&isp_ctx->ipi_occupied, 0);
> > +	atomic_set(&isp_ctx->scp_state, SCP_ON);
> > +
> > +	mutex_lock(&isp_ctx->lock);
> > +	if (!isp_ctx->composer_tx_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_tx_thread.wq);
> > +		INIT_LIST_HEAD(&isp_ctx->composer_txlist.queue);
> > +		spin_lock_init(&isp_ctx->composer_txlist.lock);
> > +		isp_ctx->composer_tx_thread.thread =
> > +			kthread_run(isp_composer_tx_work, isp_ctx,
> > +				    "isp_composer_tx");
> > +		if (IS_ERR(isp_ctx->composer_tx_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_tx_thread.thread = NULL;
> > +			goto nomem;
> 
> Why nomem?
> 

It is wrong. Need to correct with
ERR_PTR(isp_ctx->composer_tx_thread.thread).
These kthread handling will be removed in next patch.

> > +		}
> > +	} else {
> > +		dev_warn(dev, "old tx thread is existed\n");
> 
> This shouldn't be possible to happen.
> 

Yes, it should not be happen. Otherwise, there is a bug.

> > +	}
> > +
> > +	if (!isp_ctx->composer_deinit_thread.thread) {
> > +		init_waitqueue_head(&isp_ctx->composer_deinit_thread.wq);
> > +		isp_ctx->composer_deinit_thread.thread =
> > +			kthread_run(isp_composer_deinit_work, isp_ctx,
> > +				    "isp_composer_deinit_work");
> 
> Why do we need to deinit from another kthread?
> 

This code will be removed in next patch.

> > +		if (IS_ERR(isp_ctx->composer_deinit_thread.thread)) {
> > +			dev_err(dev, "unable to start kthread\n");
> > +			isp_ctx->composer_deinit_thread.thread = NULL;
> > +			goto nomem;
> > +		}
> > +	} else {
> > +		dev_warn(dev, "old rx thread is existed\n");
> 
> rx? The code above seems to refer to deinit.
> 

Got it.

> > +	}
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return 0;
> > +
> > +nomem:
> > +	mutex_unlock(&isp_ctx->lock);
> > +
> > +	return -ENOMEM;
> 
> We should return the original error code here.
> 

Got it.

> > +}
> > +
> > +void isp_composer_enqueue(struct device *dev,
> > +			  void *data,
> > +			  enum mtk_isp_scp_type type)
> > +{
> > +	struct mtk_isp_queue_work *isp_composer_work;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> 
> Just pass p1_dev to this function instead of dev.
> 

Fix in next patch.

> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	isp_composer_work = kzalloc(sizeof(*isp_composer_work), GFP_KERNEL);
> 
> For most of the cases, it should be possible to preallocate this, e.g.
> >  - for FRAME, this could be inside the request struct,
> >  - for buffer queue it could be inside the buffer struct.
> 
> I'd suggest making the caller responsible for allocating if needed.
> 

Fix in next patch.

> > +	isp_composer_work->type = type;
> > +
> > +	switch (type) {
> > +	case SCP_ISP_CMD:
> > +		memcpy(&isp_composer_work->cmd, data,
> > +		       sizeof(isp_composer_work->cmd));
> > +		dev_dbg(dev, "Enq ipi cmd id:%d\n",
> > +			isp_composer_work->cmd.cmd_id);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		atomic_inc(&isp_ctx->cmd_queued);
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > +		break;
> > +	case SCP_ISP_FRAME:
> > +		memcpy(&isp_composer_work->frameparams, data,
> > +		       sizeof(isp_composer_work->frameparams));
> > +		dev_dbg(dev, "Enq ipi frame_num:%d\n",
> > +			isp_composer_work->frameparams.frame_seq_no);
> > +
> > +		spin_lock(&isp_ctx->composer_txlist.lock);
> > +		list_add_tail(&isp_composer_work->list_entry,
> > +			      &isp_ctx->composer_txlist.queue);
> > +		atomic_inc(&isp_ctx->composer_txlist.queue_cnt);
> > +		spin_unlock(&isp_ctx->composer_txlist.lock);
> > +
> > +		wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> 
> The code in both cases is almost exactly the same. The only difference is
> the memcpy destination and size and whether isp_ctx->cmd_queued is
> incremented or not.
> 
> The memcpy will go away if my comment above is addressed and so that would
> go down to making the cmd_queued increment conditional.
> 

This function will be removed in next patch.
We will call scp_ipi_send directly in the caller, such as:

void mtk_isp_hw_config(struct mtk_cam_dev *cam,
		       struct p1_config_param *config_param)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
	memcpy(&composer_tx_cmd.config_param, config_param,
	       sizeof(*config_param));

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
	composer_tx_cmd.is_stream_on = on;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
}

static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
{
	struct mtk_isp_scp_p1_cmd composer_tx_cmd;

	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;

	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);

	isp_composer_uninit(p1_dev);
}


> > +		break;
> > +	default:
> > +		break;
> > +	}
> > +}
> > +
> > +void isp_composer_hw_init(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > +	composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > +	composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> 
> Should we also specify the size of the buffer? Otherwise we could end up
> with some undetectable overruns.
> 

The size of SCP composer's memory is fixed to 0x200000.
Is it necessary to specify the size of this buffer?

#define MTK_ISP_COMPOSER_MEM_SIZE 0x200000

ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
			MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);

> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_meta_config(struct device *dev,
> > +			      unsigned int dma)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG_META;
> > +	composer_tx_cmd.cfg_meta_out_param.enabled_meta_dmas = dma;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_config(struct device *dev,
> > +			    struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_stream(struct device *dev, int on)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +}
> > +
> > +void isp_composer_hw_deinit(struct device *dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct isp_p1_device *p1_dev = get_p1_device(dev);
> > +	struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > +	int ret;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +	isp_composer_enqueue(dev, &composer_tx_cmd, SCP_ISP_CMD);
> > +
> > +	/* Wait for ISP_CMD_DEINIT command is handled done */
> > +	ret = wait_event_timeout(isp_ctx->composer_deinit_thread.wq,
> > +				 atomic_read(&isp_ctx->scp_state) == SCP_OFF,
> > +				 msecs_to_jiffies(2000));
> > +	if (ret)
> > +		return;
> > +
> > +	dev_warn(dev, "Timeout & local de-init\n");
> > +	isp_composer_deinit(isp_ctx);
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > new file mode 100644
> > index 000000000000..fbd8593e9c2d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-scp.h
> > @@ -0,0 +1,207 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + */
> > +
> > +#ifndef _MTK_ISP_SCP_H
> > +#define _MTK_ISP_SCP_H
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam-v4l2-util.h"
> > +
> > +/*
> > + * struct img_size - image size information.
> > + *
> > + * @w: image width, the unit is pixel
> > + * @h: image height, the unit is pixel
> > + * @xsize: bytes per line based on width.
> > + * @stride: bytes per line when changing line.
> > + *          Normally, calculate new STRIDE based on
> > + *          xsize + HW constrain(page or align).
> > + *
> > + */
> > +struct img_size {
> > +	__u32 w;
> > +	__u32 h;
> > +	__u32 xsize;
> > +	__u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct img_buffer - buffer address information.
> > + *
> > + * @iova: DMA address for external devices.
> > + * @scp_addr: SCP address for external co-process unit.
> > + *
> > + */
> > +struct img_buffer {
> > +	__u32 iova;
> > +	__u32 scp_addr;
> > +} __packed;
> > +
> > +struct p1_img_crop {
> > +	__u32 left;
> > +	__u32 top;
> > +	__u32 width;
> > +	__u32 height;
> > +} __packed;
> > +
> > +struct p1_img_output {
> > +	struct img_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	__u8 pixel_byte;
> > +	__u32 img_fmt;
> > +} __packed;
> 
> Please document.
> 

Fix in next patch.

> > +
> > +/*
> > + * struct cfg_in_param - image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: indicate the sensor mode.
> > + *              1: continuous
> > + *              0: single
> > + * @subsample: indicate to enables SOF subsample or not.
> > + * @pixel_mode: describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: describe input data pattern.
> > + * @raw_pixel_id: bayer sequence.
> > + * @tg_fps: the fps rate of TG (time generator).
> > + * @img_fmt: the image format of input source.
> > + * @p1_img_crop: the crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	__u8 continuous;
> > +	__u8 subsample;
> > +	__u8 pixel_mode;
> > +	__u8 data_pattern;
> > +	__u8 raw_pixel_id;
> > +	__u16 tg_fps;
> > +	__u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - the image output parameters of main stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @pure_raw: indicate the image path control.
> > + *            1: pure raw
> > + *            0: processing raw
> > + * @pure_raw_pack: indicate the image is packed or not.
> > + *                 1: packed mode
> > + *                 0: unpacked mode
> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	/* Bypass main out parameters */
> > +	__u8 bypass;
> > +	/* Control HW image raw path */
> > +	__u8 pure_raw;
> > +	/* Control HW image pack function */
> 
> No need for these inline comments.
> 

Fix in next patch.

> > +	__u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - the image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: indicate this device is enabled or disabled or not .
> 
> Remove the space before the period.
> 

Fix in next patch.

> > + * @p1_img_output: the output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	/* Bypass resize parameters */
> 
> No need for this inline comment.
> 

Fix in next patch.

> > +	__u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_meta_out_param - output meta information.
> > + *
> > + * @enabled_meta_dmas: indicate which meta DMAs are enabled.
> > + *
> > + */
> > +struct cfg_meta_out_param {
> > +	__u32 enabled_meta_dmas;
> > +} __packed;
> > +
> > +struct p1_config_param {
> > +	/* Sensor/TG info */
> > +	struct cfg_in_param cfg_in_param;
> > +	/* IMGO DMA */
> > +	struct cfg_main_out_param cfg_main_param;
> > +	/* RRZO DMA */
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	/* 3A DMAs and other. */
> > +	struct cfg_meta_out_param cfg_meta_param;
> 
> Please change the inline comments to a kerneldoc comment at the top.
> 

Fix in next patch.

> > +} __packed;
> > +
> > +struct p1_frame_param {
> > +	/* frame sequence number */
> > +	__u32 frame_seq_no;
> > +	/* SOF index */
> > +	__u32 sof_idx;
> > +	/* The memory address of tuning buffer from user space */
> 
> Ditto.
> 

Fix in next patch.

> > +	struct img_buffer dma_buffers[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +struct P1_meta_frame {
> > +	__u32 enabled_dma;
> > +	__u32 vb_index;
> > +	struct img_buffer meta_addr;
> > +} __packed;
> > +
> > +struct isp_init_info {
> > +	__u8 hw_module;
> > +	struct img_buffer cq_addr;
> > +} __packed;
> > +
> > +struct isp_ack_info {
> > +	__u8 cmd_id;
> > +	__u32 frame_seq_no;
> > +} __packed;
> > +
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_CONFIG_META,
> > +	ISP_CMD_ENQUEUE_META,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +struct mtk_isp_scp_p1_cmd {
> > +	__u8 cmd_id;
> > +	union {
> > +		struct isp_init_info frameparam;
> > +		struct p1_config_param config_param;
> > +		struct cfg_meta_out_param cfg_meta_out_param;
> > +		struct P1_meta_frame meta_frame;
> > +		__u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +enum mtk_isp_scp_type {
> > +	SCP_ISP_CMD = 0,
> > +	SCP_ISP_FRAME,
> > +};
> 
> Please document all the structs and enum above using kerneldoc.
> 

Fix in next patch.

> Best regards,
> Tomasz
> 

Thank you for your valuable comments.

Best regards,


Jungo 



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-05  7:59               ` Jungo Lin
  (?)
@ 2019-07-23  7:20                 ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23  7:20 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Frederic Chen (陳俊元)

Hi Jungo,

On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
>
> Hi Tomasz:
>
> On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > >
> > > Hi Tomasz,
>
> [snip]
>
> > > After applying your suggestion in SCP device driver, we could remove
> > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > to get SCP address. We could touch the buffer with this SCP address in
> > > SCP processor.
> > >
> > > After that, we use dma_map_page_attrs with P1 device which supports
> > > IOMMU domain to get IOVA address. For this address, we will assign
> > > it to our ISP HW device to proceed.
> > >
> > > Below is the snippet for ISP P1 compose buffer initialization.
> > >
> > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > >         if (!ptr) {
> > >                 dev_err(dev, "failed to allocate compose memory\n");
> > >                 return -ENOMEM;
> > >         }
> > >         isp_ctx->scp_mem_pa = addr;
> >
> > addr contains a DMA address, not a physical address. Could we call it
> > scp_mem_dma instead?
> >
> > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > >
> > >         /* get iova address */
> > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> >
> > addr is a DMA address, so phys_to_page() can't be called on it. The
> > simplest thing here would be to use dma_map_single() with ptr as the
> > CPU address expected.
> >
>
> We have changed to use ma_map_single() with ptr, but encounter IOMMU
> error. From the debug log of iommu_dma_map_page[3], we got
> 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> There is a address offset(0x4000000). If we change to use
> dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> expected[2]. Do you have any suggestion on this issue? Do we miss
> something?

Sorry for the late reply. Could you show me the code changes you made
to use dma_map_single()? It would sound like the virtual address
passed to dma_map_single() isn't correct.

Best regards,
Tomasz

>
> [1]
> [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
>
> [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
>
> [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
> [2]
> [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
> [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
> [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
>
> [3]
> dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
>                 unsigned long offset, size_t size, int prot)
> {
>         phys_addr_t phys = page_to_phys(page);
>         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
>
>         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
>                         iommu_get_dma_domain(dev));
> }
>
> [snip]
>
> Best regards,
>
> Jungo
>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-23  7:20                 ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23  7:20 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi Jungo,

On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Tomasz:
>
> On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi Tomasz,
>
> [snip]
>
> > > After applying your suggestion in SCP device driver, we could remove
> > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > to get SCP address. We could touch the buffer with this SCP address in
> > > SCP processor.
> > >
> > > After that, we use dma_map_page_attrs with P1 device which supports
> > > IOMMU domain to get IOVA address. For this address, we will assign
> > > it to our ISP HW device to proceed.
> > >
> > > Below is the snippet for ISP P1 compose buffer initialization.
> > >
> > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > >         if (!ptr) {
> > >                 dev_err(dev, "failed to allocate compose memory\n");
> > >                 return -ENOMEM;
> > >         }
> > >         isp_ctx->scp_mem_pa = addr;
> >
> > addr contains a DMA address, not a physical address. Could we call it
> > scp_mem_dma instead?
> >
> > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > >
> > >         /* get iova address */
> > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> >
> > addr is a DMA address, so phys_to_page() can't be called on it. The
> > simplest thing here would be to use dma_map_single() with ptr as the
> > CPU address expected.
> >
>
> We have changed to use ma_map_single() with ptr, but encounter IOMMU
> error. From the debug log of iommu_dma_map_page[3], we got
> 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> There is a address offset(0x4000000). If we change to use
> dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> expected[2]. Do you have any suggestion on this issue? Do we miss
> something?

Sorry for the late reply. Could you show me the code changes you made
to use dma_map_single()? It would sound like the virtual address
passed to dma_map_single() isn't correct.

Best regards,
Tomasz

>
> [1]
> [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
>
> [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
>
> [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
> [2]
> [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
> [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
> [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
>
> [3]
> dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
>                 unsigned long offset, size_t size, int prot)
> {
>         phys_addr_t phys = page_to_phys(page);
>         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
>
>         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
>                         iommu_get_dma_domain(dev));
> }
>
> [snip]
>
> Best regards,
>
> Jungo
>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-23  7:20                 ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23  7:20 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Tomasz:
>
> On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi Tomasz,
>
> [snip]
>
> > > After applying your suggestion in SCP device driver, we could remove
> > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > to get SCP address. We could touch the buffer with this SCP address in
> > > SCP processor.
> > >
> > > After that, we use dma_map_page_attrs with P1 device which supports
> > > IOMMU domain to get IOVA address. For this address, we will assign
> > > it to our ISP HW device to proceed.
> > >
> > > Below is the snippet for ISP P1 compose buffer initialization.
> > >
> > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > >         if (!ptr) {
> > >                 dev_err(dev, "failed to allocate compose memory\n");
> > >                 return -ENOMEM;
> > >         }
> > >         isp_ctx->scp_mem_pa = addr;
> >
> > addr contains a DMA address, not a physical address. Could we call it
> > scp_mem_dma instead?
> >
> > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > >
> > >         /* get iova address */
> > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> >
> > addr is a DMA address, so phys_to_page() can't be called on it. The
> > simplest thing here would be to use dma_map_single() with ptr as the
> > CPU address expected.
> >
>
> We have changed to use ma_map_single() with ptr, but encounter IOMMU
> error. From the debug log of iommu_dma_map_page[3], we got
> 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> There is a address offset(0x4000000). If we change to use
> dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> expected[2]. Do you have any suggestion on this issue? Do we miss
> something?

Sorry for the late reply. Could you show me the code changes you made
to use dma_map_single()? It would sound like the virtual address
passed to dma_map_single() isn't correct.

Best regards,
Tomasz

>
> [1]
> [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
>
> [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
>
> [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
> [2]
> [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> device_base:0x0000000050000000 dma:0x0000000050800000
> virt_base:ffffff8014000000 va:ffffff8014800000
> [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> va:ffffff8014800000
> [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
>
>
> [3]
> dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
>                 unsigned long offset, size_t size, int prot)
> {
>         phys_addr_t phys = page_to_phys(page);
>         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
>
>         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
>                         iommu_get_dma_domain(dev));
> }
>
> [snip]
>
> Best regards,
>
> Jungo
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-23  7:20                 ` [RFC,v3 " Tomasz Figa
  (?)
@ 2019-07-23  8:21                     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-23  8:21 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport-F7+t8E8rja9g9hUCZPvPmw, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab, list-Y9sIeH5OGRo

Hi, Tomasz:

On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> >
> > Hi Tomasz:
> >
> > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > > >
> > > > Hi Tomasz,
> >
> > [snip]
> >
> > > > After applying your suggestion in SCP device driver, we could remove
> > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > SCP processor.
> > > >
> > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > it to our ISP HW device to proceed.
> > > >
> > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > >
> > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > >         if (!ptr) {
> > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > >                 return -ENOMEM;
> > > >         }
> > > >         isp_ctx->scp_mem_pa = addr;
> > >
> > > addr contains a DMA address, not a physical address. Could we call it
> > > scp_mem_dma instead?
> > >
> > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > >
> > > >         /* get iova address */
> > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > >
> > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > simplest thing here would be to use dma_map_single() with ptr as the
> > > CPU address expected.
> > >
> >
> > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > error. From the debug log of iommu_dma_map_page[3], we got
> > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > There is a address offset(0x4000000). If we change to use
> > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > expected[2]. Do you have any suggestion on this issue? Do we miss
> > something?
> 
> Sorry for the late reply. Could you show me the code changes you made
> to use dma_map_single()? It would sound like the virtual address
> passed to dma_map_single() isn't correct.
> 
> Best regards,
> Tomasz
> 


Please check the below code snippet in today's testing.

	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/* get iova address */
	addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
DMA_BIDIRECTIONAL);
	if (dma_mapping_error(dev, addr)) {
		dma_free_coherent(p1_dev->cam_dev.smem_dev,
				  MTK_ISP_COMPOSER_MEM_SIZE,
				  ptr, p1_dev->composer_scp_addr);
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, below is extracted log[2].

We guess the virtual address which is returned by dma_alloc_coherent
function is not valid kernel logical address. It is actually returned by
memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
will call virt_to_page() function. For virt_to_page function, it
requires a logical address[1].

[1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html

[2]
  322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:00000000a3adc471
  323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
 7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:0000000042ec580f
 7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:000000005e5b3462
15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000

Best regards,

Jungo

> >
> > [1]
> > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> >
> > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> >
> > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> > [2]
> > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> >
> > [3]
> > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> >                 unsigned long offset, size_t size, int prot)
> > {
> >         phys_addr_t phys = page_to_phys(page);
> >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> >
> >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> >                         iommu_get_dma_domain(dev));
> > }
> >
> > [snip]
> >
> > Best regards,
> >
> > Jungo
> >
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-23  8:21                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-23  8:21 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz:
> >
> > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi Tomasz,
> >
> > [snip]
> >
> > > > After applying your suggestion in SCP device driver, we could remove
> > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > SCP processor.
> > > >
> > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > it to our ISP HW device to proceed.
> > > >
> > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > >
> > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > >         if (!ptr) {
> > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > >                 return -ENOMEM;
> > > >         }
> > > >         isp_ctx->scp_mem_pa = addr;
> > >
> > > addr contains a DMA address, not a physical address. Could we call it
> > > scp_mem_dma instead?
> > >
> > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > >
> > > >         /* get iova address */
> > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > >
> > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > simplest thing here would be to use dma_map_single() with ptr as the
> > > CPU address expected.
> > >
> >
> > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > error. From the debug log of iommu_dma_map_page[3], we got
> > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > There is a address offset(0x4000000). If we change to use
> > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > expected[2]. Do you have any suggestion on this issue? Do we miss
> > something?
> 
> Sorry for the late reply. Could you show me the code changes you made
> to use dma_map_single()? It would sound like the virtual address
> passed to dma_map_single() isn't correct.
> 
> Best regards,
> Tomasz
> 


Please check the below code snippet in today's testing.

	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/* get iova address */
	addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
DMA_BIDIRECTIONAL);
	if (dma_mapping_error(dev, addr)) {
		dma_free_coherent(p1_dev->cam_dev.smem_dev,
				  MTK_ISP_COMPOSER_MEM_SIZE,
				  ptr, p1_dev->composer_scp_addr);
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, below is extracted log[2].

We guess the virtual address which is returned by dma_alloc_coherent
function is not valid kernel logical address. It is actually returned by
memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
will call virt_to_page() function. For virt_to_page function, it
requires a logical address[1].

[1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html

[2]
  322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:00000000a3adc471
  323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
 7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:0000000042ec580f
 7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:000000005e5b3462
15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000

Best regards,

Jungo

> >
> > [1]
> > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> >
> > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> >
> > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> > [2]
> > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> >
> > [3]
> > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> >                 unsigned long offset, size_t size, int prot)
> > {
> >         phys_addr_t phys = page_to_phys(page);
> >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> >
> >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> >                         iommu_get_dma_domain(dev));
> > }
> >
> > [snip]
> >
> > Best regards,
> >
> > Jungo
> >
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-23  8:21                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-23  8:21 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi Tomasz:
> >
> > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi Tomasz,
> >
> > [snip]
> >
> > > > After applying your suggestion in SCP device driver, we could remove
> > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > SCP processor.
> > > >
> > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > it to our ISP HW device to proceed.
> > > >
> > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > >
> > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > >         if (!ptr) {
> > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > >                 return -ENOMEM;
> > > >         }
> > > >         isp_ctx->scp_mem_pa = addr;
> > >
> > > addr contains a DMA address, not a physical address. Could we call it
> > > scp_mem_dma instead?
> > >
> > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > >
> > > >         /* get iova address */
> > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > >
> > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > simplest thing here would be to use dma_map_single() with ptr as the
> > > CPU address expected.
> > >
> >
> > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > error. From the debug log of iommu_dma_map_page[3], we got
> > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > There is a address offset(0x4000000). If we change to use
> > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > expected[2]. Do you have any suggestion on this issue? Do we miss
> > something?
> 
> Sorry for the late reply. Could you show me the code changes you made
> to use dma_map_single()? It would sound like the virtual address
> passed to dma_map_single() isn't correct.
> 
> Best regards,
> Tomasz
> 


Please check the below code snippet in today's testing.

	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/* get iova address */
	addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
DMA_BIDIRECTIONAL);
	if (dma_mapping_error(dev, addr)) {
		dma_free_coherent(p1_dev->cam_dev.smem_dev,
				  MTK_ISP_COMPOSER_MEM_SIZE,
				  ptr, p1_dev->composer_scp_addr);
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, below is extracted log[2].

We guess the virtual address which is returned by dma_alloc_coherent
function is not valid kernel logical address. It is actually returned by
memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
will call virt_to_page() function. For virt_to_page function, it
requires a logical address[1].

[1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html

[2]
  322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:00000000a3adc471
  323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
 7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:0000000042ec580f
 7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000
15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
addr:0x0000000052000000 va:000000005e5b3462
15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
addr:0x00000000fde00000

Best regards,

Jungo

> >
> > [1]
> > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> >
> > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> >
> > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> > [2]
> > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > device_base:0x0000000050000000 dma:0x0000000050800000
> > virt_base:ffffff8014000000 va:ffffff8014800000
> > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > va:ffffff8014800000
> > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> >
> >
> > [3]
> > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> >                 unsigned long offset, size_t size, int prot)
> > {
> >         phys_addr_t phys = page_to_phys(page);
> >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> >
> >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> >                         iommu_get_dma_domain(dev));
> > }
> >
> > [snip]
> >
> > Best regards,
> >
> > Jungo
> >
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-18  4:39         ` Jungo Lin
  (?)
@ 2019-07-23 10:21           ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23 10:21 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab

Hi Jungo,

On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > +                                 struct media_request *new_req)
> > > +{
> > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +
> > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > +
> > > +   if (!cam_dev->streaming) {
> > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > +           spin_lock(&cam_dev->req_lock);
> > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > +           spin_unlock(&cam_dev->req_lock);
> > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Normal enqueue flow */
> > > +   if (new_req) {
> > > +           mtk_isp_req_enqueue(dev, new_req);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Flush all media requests wehen first stream on */
> > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > +           list_del(&req->list);
> > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > +   }
> > > +}
> >
> > This will have to be redone, as per the other suggestions, but generally one
> > would have a function that tries to queue as much as possible from a list to
> > the hardware and another function that adds a request to the list and calls
> > the first function.
> >
>
> We revised this function as below.
> First to check the en-queue conditions:
> a. stream on
> b. The composer buffers in SCP are 3, so we only could has 3 jobs
> at the same time.
>
>
> Second, try to en-queue the frames in the pending job if possible and
> move them into running job list if possible.
>
> The request has been inserted into pending job in mtk_cam_req_validate
> which is used to validate media_request.

Thanks for replying to each of the comments, that's very helpful.
Snipped out the parts that I agreed with.

Please note that req_validate is not supposed to change any driver
state. It's only supposed to validate the request. req_queue is the
right callback to insert the request into some internal driver
bookkeeping structures.

>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         struct list_head enqueue_job_list;
>         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
>         unsigned long flags;
>
>         if (!cam_dev->streaming ||
>             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {

Do we have a guarantee that cam_dev->running_job_count doesn't
decrement between the atomic_read() above and this line?

>                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
>                         buffer_cnt);
>                 return;
>         }
>
>         INIT_LIST_HEAD(&enqueue_job_list);
>
>         spin_lock(&cam_dev->pending_job_lock);
>         list_for_each_entry_safe(req, req_prev,
>                                  &cam_dev->pending_job_list, list) {
>                 list_del(&req->list);
>                 list_add_tail(&req->list, &enqueue_job_list);

What's the reason to use the second list? Could we just take one job
from pending_job_list, enqueue it and then iterate again?

>                 if (atomic_inc_return(&cam_dev->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;
>         }
>         spin_unlock(&cam_dev->pending_job_lock);
>
>         list_for_each_entry_safe(req, req_prev,
>                                  &enqueue_job_list, list) {
>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam_dev->running_job_list);
>                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
>

Do we have a guarantee that another thread doesn't run the same
function ending up calling mtk_isp_req_enqueue() with another request
before this one and thus making the order of running_job_list
incorrect?

>                 mtk_isp_req_enqueue(cam_dev, req);
>         }
> }
>
[snip]
> > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > +
> > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > +           stride = ALIGN(stride, 4);
> >
> > Is it expected that only the F10 format needs this alignment?
> >
>
> yes, if the pixel bits of image format is 10, the byte alignment of bpl
> should be 4. Otherwise, it is 8. We will revise this and add more
> comments.

That means that the B10 format also needs the extra alignment, as
opposed to what the original code did, right?

>
> /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
>         if (pixel_bits == 10)
>                 bpl = ALIGN(bpl, 4);
>         else
>                 bpl = ALIGN(bpl, 8);

SGTM, thanks.

[snip]
> > > +
> > > +static struct v4l2_subdev *
> > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > +   struct media_entity *entity;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct v4l2_subdev *sensor;
> >
> > This variable would be unitialized if there is no streaming sensor. Was
> > there no compiler warning generated for this?
> >
>
> No, there is no compiler warning.
> But, we will assign sensor to NULL to avoid unnecessary compiler warning
> with different compiler options.
>

Thanks. It would be useful if you could check why the compiler you're
using doesn't show a warning here. We might be missing other
uninitialized variables.

[snip]
> > > +}
> > > +
> > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > +                               const struct media_pad *local,
> > > +                               const struct media_pad *remote, u32 flags)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > +   u32 pad = local->index;
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > +           __func__, pad, remote->index, flags);
> > > +
> > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> >
> > I assume this check is needed, because the pads with higher indexes are not
> > video nodes? If so, a comment would be helpful here.
> >
>
> Yes, we will new comment as below.
>
>         /*
>          * Check video nodes is enabled by link setup.
>          * The pad index of video node should be less than
>          * MTK_CAM_P1_TOTAL_NODES.
>          */
>         if (pad < MTK_CAM_P1_TOTAL_NODES)
>                 cam_dev->vdev_nodes[pad].enabled =
>                         !!(flags & MEDIA_LNK_FL_ENABLED);
>

Could we rephrase this a bit. The comment still doesn't explain why
the index should be less than the constant. Perhaps:

/*
 * The video nodes exposed by the driver have pads indexes
 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
 */

[snip]

> > > +
> > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > +           __func__,
> > > +           node->id,
> > > +           buf->vbb.request_fd,
> > > +           buf->vbb.vb2_buf.index);
> > > +
> > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > +   if (vb->vb2_queue->uses_requests)
> > > +           return;
> >
> > I'd suggest removing non-request support from this driver. Even if we end up
> > with a need to provide compatibility for non-request mode, then it should be
> > built on top of the requests mode, so that the driver itself doesn't have to
> > deal with two modes.
> >
>
> The purpose of non-request function in this driver is needed by
> our camera middle-ware design. It needs 3A statistics buffers before
> image buffers en-queue. So we need to en-queue 3A statistics with
> non-request mode in this driver. After MW got the 3A statistics data, it
> will en-queue the images, tuning buffer and other meta buffers with
> request mode. Based on this requirement, do you have any suggestion?
> For upstream driver, should we only consider request mode?
>

Where does that requirement come from? Why the timing of queuing of
the buffers to the driver is important?

[snip]
> > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > +                                  unsigned int count)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > +   int ret;
> > > +
> > > +   if (!node->enabled) {
> >
> > How is this synchronized with mtk_cam_media_link_setup()?
> >
>
> We will follow your suggestion and below is our proposal for this
> function.
>
> 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> stream-on.
> 2.a If yes, do the following steps
>     2.a-1 Call media_pipeline_start function to prevent the link
> configuration changes.
>     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> video nodes are enabled and save it into cam_dev->enabled_node_count.
>     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> user has called stream-on API
> 2.b jump step 3.
>
> 3. Use cam_dev->streamed_node_count to track how many video nodes are
> streamed by user space.
> 4. Check all enabled video nodes are streamed or not based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 5. If yes, call s_stream on for P1 sub-device
>
> Do you think it is reasonable?
>

That should work indeed.

[snip]
> > > +
> > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> >
> > Shouldn't we stop streaming first, so that the hardware operation is
> > cancelled and any buffers owned by the hardware are released?
> >
>
> For this function, below is the new code flow.
>
> 1. Check the first node to stream off based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 2. If yes, call all s_stream off for P1 sub-device
> 3. Call mtk_cam_vb2_return_all_buffers for each node
> 4. Check the last node to stream off
> 5. If yes, call media_pipeline_stop to allow user space
> to perform link configuration changes, such as disable link.
>
> But, for step 5, is it too late for end user to disable link?
> For example, for first node, it has called stream off but
> can't call disable link until the last node is stream off?
>

I think that should be okay. From the userspace point of view, having
one of the video nodes streaming implies that the related subdevice
could be streaming as well. The links between the video nodes and the
subdevices don't have the DYNAMIC flag, so the userspace should expect
that it can't change any links connecting to the subdevice when the
subdevice could be streaming.

[snip]
> > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > +{
> > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > +
> > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > +                              dev->v4l2_dev.ctrl_handler);
> >
> > This would end up being called multiple times, once for each video node.
> > Instead, this should be called explicitly by the driver when it completed
> > the request - perhaps in the frame completion handler?
> >
> > With that, we probably wouldn't even need this callback.
> >
>
> First, if we don't implement this callback function, we will receive
> kernel warning as below.
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
>
> Second, this function is only be called in __vb2_queue_cancel function.
> Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> So could we just implement dummy empty function?
>
>  * @buf_request_complete: a buffer that was never queued to the driver
> but is
>  *                      associated with a queued request was canceled.
>  *                      The driver will have to mark associated objects in the
>  *                      request as completed; required if requests are
>  *                      supported.
>

Good catch, thanks.

Sounds like we may indeed need to implement this callback. In
particular, we may need to remove the request that the buffer was
associated with from the driver queue and return the other buffers
associated to it with an error state. This should be similar to
handling a request failure.
[snip]
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > +                              struct v4l2_fmtdesc *f)
> > > +{
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (f->index >= node->desc.num_fmts)
> > > +           return -EINVAL;
> > > +
> > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >
> > Is the set of formats available always the same regardless of the sensor
> > format?
> >
>
> Yes, ISP P1 HW output formats are always available without impact
> by sensor formats.
>
> > > +   f->flags = 0;
> >
> > We need f->description too.
> >
>
> For this description, do you suggest 1). we fill this field in this
> function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
>
> Basically, we prefer method 1.
>

That should be v4l_fill_fmtdesc(), as it already includes other
vendor-specific formats.

[snip]
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > +           __func__,
> > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > +
> > > +   width = in_fmt->fmt.pix_mp.width;
> > > +   height = in_fmt->fmt.pix_mp.height;
> > > +
> > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > +   if (dev_fmt) {
> > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > +                                   &in_fmt->fmt.pix_mp,
> > > +                                   &dev_fmt->fmt.pix_mp,
> > > +                                   node->id);
> > > +   } else {
> > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > +                                        &node->desc, in_fmt);
> >
> > We shouldn't just load a default format. This function should validate all
> > the fields one by one and adjust them to something appropriate.
> >
>
> For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> a. width
> b. height
> c. pixelformat
> d. plane_fmt
>     - sizeimage
>     - bytesperline
> e. num_planes
> Other fields are consider constant.
>
> So if the user space passes one pixel format with un-supported, we will
> apply the default format firstly and adjust width, height, sizeimage,
> and bytesperline. We will focus on validate width & height.
> Is it ok?

I'm not sure I understand your proposal, but let me describe the
proper behavior here:

if (pixelformat is invalid)
    pixelformat = some valid pixel format;

width = clamp(width, driver min, driver max);
height = clamp(height, driver min, driver max);

num_planes = 1;

calculate_sizeimage_and_bytesperline(fmt);

fill_in_the_remaining_constant_fields(fmt);

Does it make sense?

[snip]
> > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > +                           struct v4l2_format *f)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (cam_dev->streaming)
> > > +           return -EBUSY;
> >
> > I think this should rather be something like vb2_queue_is_busy(), which
> > would prevent format changes if buffers are allocated.
> >
>
> Since vb2_queue_is_busy is static function, would we paste its
> implementation in this function to check like this?
>
>         if (node->vdev.queue->owner &&
>                 node->vdev.queue->owner != file->private_data) {
>                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
>                 return -EBUSY;
>         }
>

Sorry, I mixed up the function name. That should've been vb2_is_busy().

[snip]
> > > +   /* Total pad numbers is video devices + one seninf pad */
> > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > +   unsigned int i;
> > > +   int ret;
> > > +
> > > +   ret = mtk_cam_media_register(dev,
> > > +                                &cam_dev->media_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > +           return ret;
> > > +   }
> > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> >
> > An info message should be useful to the user in some way. Printing kernel
> > pointers isn't useful. Something like "registered media0" could be useful to
> > let the user know which media device is associated with this driver if there
> > is more than one in the system.
> >
>
> Here is the new log info.
>
> dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
>

Let's fix the missing space and making a bit more readable:

dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);

>
> > > +
> > > +   /* Set up v4l2 device */
> > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > +           goto fail_v4l2_dev;
> >
> > Please call the labels after the cleanup step that needs to be done. It
> > makes it easier to spot any ordering errors.
> >
>
> Will fix in next patch.
>
> > > +   }
> > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> >
> > Same as above.
> >
>
> Ditto.
>
> dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
>

Perhaps just "Registered %s" to be consistent with the above media log?

[snip]
> > > +
> > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > +                                 struct v4l2_subdev *sd,
> > > +                                 struct v4l2_async_subdev *asd)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > +
> >
> > Should we somehow check that the entity we got is seninf indeed and there
> > was no mistake in DT?
> >
>
> How about to check the entity function of seninf device?
>
> if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>                 return -ENODEV;
> }
>
> If we need to check DT, may we need to implement this in parse_endpoint
> callback function of v4l2_async_notifier_parse_fwnode_endpoints?
>

Yes, checking the entity function is indeed the right way to do this.

[snip]
> > > +           .default_fmt_idx = 4,
> > > +           .max_buf_count = 10,
> >
> > Where does this number come from?
> >
>
> The default maximum VB2 buffer count is 32.
> In order to limit memory usage, we like to limit the maximum buffer
> counts in the driver layer. The maximum buffer count is estimated
> according to our camera MW.
>
> #define VB2_MAX_FRAME   (32)
>

Okay, thanks.

[snip]
> > > +   struct media_pad vdev_pad;
> > > +   struct vb2_queue vbq;
> > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > +   struct list_head pending_list;
> > > +   /* Used for vbq & vdev */
> >
> > It's already documented in the kerneldoc comment.
> >
>
> Fixed in next patch.
> Btw, if we remove this, we will got complain from checkpatch.pl script.
>

Oh really, that's weird. Okay, please keep it then, sorry for the confusion.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-23 10:21           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23 10:21 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi Jungo,

On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > +                                 struct media_request *new_req)
> > > +{
> > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +
> > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > +
> > > +   if (!cam_dev->streaming) {
> > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > +           spin_lock(&cam_dev->req_lock);
> > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > +           spin_unlock(&cam_dev->req_lock);
> > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Normal enqueue flow */
> > > +   if (new_req) {
> > > +           mtk_isp_req_enqueue(dev, new_req);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Flush all media requests wehen first stream on */
> > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > +           list_del(&req->list);
> > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > +   }
> > > +}
> >
> > This will have to be redone, as per the other suggestions, but generally one
> > would have a function that tries to queue as much as possible from a list to
> > the hardware and another function that adds a request to the list and calls
> > the first function.
> >
>
> We revised this function as below.
> First to check the en-queue conditions:
> a. stream on
> b. The composer buffers in SCP are 3, so we only could has 3 jobs
> at the same time.
>
>
> Second, try to en-queue the frames in the pending job if possible and
> move them into running job list if possible.
>
> The request has been inserted into pending job in mtk_cam_req_validate
> which is used to validate media_request.

Thanks for replying to each of the comments, that's very helpful.
Snipped out the parts that I agreed with.

Please note that req_validate is not supposed to change any driver
state. It's only supposed to validate the request. req_queue is the
right callback to insert the request into some internal driver
bookkeeping structures.

>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         struct list_head enqueue_job_list;
>         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
>         unsigned long flags;
>
>         if (!cam_dev->streaming ||
>             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {

Do we have a guarantee that cam_dev->running_job_count doesn't
decrement between the atomic_read() above and this line?

>                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
>                         buffer_cnt);
>                 return;
>         }
>
>         INIT_LIST_HEAD(&enqueue_job_list);
>
>         spin_lock(&cam_dev->pending_job_lock);
>         list_for_each_entry_safe(req, req_prev,
>                                  &cam_dev->pending_job_list, list) {
>                 list_del(&req->list);
>                 list_add_tail(&req->list, &enqueue_job_list);

What's the reason to use the second list? Could we just take one job
from pending_job_list, enqueue it and then iterate again?

>                 if (atomic_inc_return(&cam_dev->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;
>         }
>         spin_unlock(&cam_dev->pending_job_lock);
>
>         list_for_each_entry_safe(req, req_prev,
>                                  &enqueue_job_list, list) {
>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam_dev->running_job_list);
>                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
>

Do we have a guarantee that another thread doesn't run the same
function ending up calling mtk_isp_req_enqueue() with another request
before this one and thus making the order of running_job_list
incorrect?

>                 mtk_isp_req_enqueue(cam_dev, req);
>         }
> }
>
[snip]
> > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > +
> > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > +           stride = ALIGN(stride, 4);
> >
> > Is it expected that only the F10 format needs this alignment?
> >
>
> yes, if the pixel bits of image format is 10, the byte alignment of bpl
> should be 4. Otherwise, it is 8. We will revise this and add more
> comments.

That means that the B10 format also needs the extra alignment, as
opposed to what the original code did, right?

>
> /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
>         if (pixel_bits == 10)
>                 bpl = ALIGN(bpl, 4);
>         else
>                 bpl = ALIGN(bpl, 8);

SGTM, thanks.

[snip]
> > > +
> > > +static struct v4l2_subdev *
> > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > +   struct media_entity *entity;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct v4l2_subdev *sensor;
> >
> > This variable would be unitialized if there is no streaming sensor. Was
> > there no compiler warning generated for this?
> >
>
> No, there is no compiler warning.
> But, we will assign sensor to NULL to avoid unnecessary compiler warning
> with different compiler options.
>

Thanks. It would be useful if you could check why the compiler you're
using doesn't show a warning here. We might be missing other
uninitialized variables.

[snip]
> > > +}
> > > +
> > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > +                               const struct media_pad *local,
> > > +                               const struct media_pad *remote, u32 flags)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > +   u32 pad = local->index;
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > +           __func__, pad, remote->index, flags);
> > > +
> > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> >
> > I assume this check is needed, because the pads with higher indexes are not
> > video nodes? If so, a comment would be helpful here.
> >
>
> Yes, we will new comment as below.
>
>         /*
>          * Check video nodes is enabled by link setup.
>          * The pad index of video node should be less than
>          * MTK_CAM_P1_TOTAL_NODES.
>          */
>         if (pad < MTK_CAM_P1_TOTAL_NODES)
>                 cam_dev->vdev_nodes[pad].enabled =
>                         !!(flags & MEDIA_LNK_FL_ENABLED);
>

Could we rephrase this a bit. The comment still doesn't explain why
the index should be less than the constant. Perhaps:

/*
 * The video nodes exposed by the driver have pads indexes
 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
 */

[snip]

> > > +
> > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > +           __func__,
> > > +           node->id,
> > > +           buf->vbb.request_fd,
> > > +           buf->vbb.vb2_buf.index);
> > > +
> > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > +   if (vb->vb2_queue->uses_requests)
> > > +           return;
> >
> > I'd suggest removing non-request support from this driver. Even if we end up
> > with a need to provide compatibility for non-request mode, then it should be
> > built on top of the requests mode, so that the driver itself doesn't have to
> > deal with two modes.
> >
>
> The purpose of non-request function in this driver is needed by
> our camera middle-ware design. It needs 3A statistics buffers before
> image buffers en-queue. So we need to en-queue 3A statistics with
> non-request mode in this driver. After MW got the 3A statistics data, it
> will en-queue the images, tuning buffer and other meta buffers with
> request mode. Based on this requirement, do you have any suggestion?
> For upstream driver, should we only consider request mode?
>

Where does that requirement come from? Why the timing of queuing of
the buffers to the driver is important?

[snip]
> > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > +                                  unsigned int count)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > +   int ret;
> > > +
> > > +   if (!node->enabled) {
> >
> > How is this synchronized with mtk_cam_media_link_setup()?
> >
>
> We will follow your suggestion and below is our proposal for this
> function.
>
> 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> stream-on.
> 2.a If yes, do the following steps
>     2.a-1 Call media_pipeline_start function to prevent the link
> configuration changes.
>     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> video nodes are enabled and save it into cam_dev->enabled_node_count.
>     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> user has called stream-on API
> 2.b jump step 3.
>
> 3. Use cam_dev->streamed_node_count to track how many video nodes are
> streamed by user space.
> 4. Check all enabled video nodes are streamed or not based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 5. If yes, call s_stream on for P1 sub-device
>
> Do you think it is reasonable?
>

That should work indeed.

[snip]
> > > +
> > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> >
> > Shouldn't we stop streaming first, so that the hardware operation is
> > cancelled and any buffers owned by the hardware are released?
> >
>
> For this function, below is the new code flow.
>
> 1. Check the first node to stream off based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 2. If yes, call all s_stream off for P1 sub-device
> 3. Call mtk_cam_vb2_return_all_buffers for each node
> 4. Check the last node to stream off
> 5. If yes, call media_pipeline_stop to allow user space
> to perform link configuration changes, such as disable link.
>
> But, for step 5, is it too late for end user to disable link?
> For example, for first node, it has called stream off but
> can't call disable link until the last node is stream off?
>

I think that should be okay. From the userspace point of view, having
one of the video nodes streaming implies that the related subdevice
could be streaming as well. The links between the video nodes and the
subdevices don't have the DYNAMIC flag, so the userspace should expect
that it can't change any links connecting to the subdevice when the
subdevice could be streaming.

[snip]
> > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > +{
> > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > +
> > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > +                              dev->v4l2_dev.ctrl_handler);
> >
> > This would end up being called multiple times, once for each video node.
> > Instead, this should be called explicitly by the driver when it completed
> > the request - perhaps in the frame completion handler?
> >
> > With that, we probably wouldn't even need this callback.
> >
>
> First, if we don't implement this callback function, we will receive
> kernel warning as below.
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
>
> Second, this function is only be called in __vb2_queue_cancel function.
> Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> So could we just implement dummy empty function?
>
>  * @buf_request_complete: a buffer that was never queued to the driver
> but is
>  *                      associated with a queued request was canceled.
>  *                      The driver will have to mark associated objects in the
>  *                      request as completed; required if requests are
>  *                      supported.
>

Good catch, thanks.

Sounds like we may indeed need to implement this callback. In
particular, we may need to remove the request that the buffer was
associated with from the driver queue and return the other buffers
associated to it with an error state. This should be similar to
handling a request failure.
[snip]
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > +                              struct v4l2_fmtdesc *f)
> > > +{
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (f->index >= node->desc.num_fmts)
> > > +           return -EINVAL;
> > > +
> > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >
> > Is the set of formats available always the same regardless of the sensor
> > format?
> >
>
> Yes, ISP P1 HW output formats are always available without impact
> by sensor formats.
>
> > > +   f->flags = 0;
> >
> > We need f->description too.
> >
>
> For this description, do you suggest 1). we fill this field in this
> function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
>
> Basically, we prefer method 1.
>

That should be v4l_fill_fmtdesc(), as it already includes other
vendor-specific formats.

[snip]
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > +           __func__,
> > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > +
> > > +   width = in_fmt->fmt.pix_mp.width;
> > > +   height = in_fmt->fmt.pix_mp.height;
> > > +
> > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > +   if (dev_fmt) {
> > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > +                                   &in_fmt->fmt.pix_mp,
> > > +                                   &dev_fmt->fmt.pix_mp,
> > > +                                   node->id);
> > > +   } else {
> > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > +                                        &node->desc, in_fmt);
> >
> > We shouldn't just load a default format. This function should validate all
> > the fields one by one and adjust them to something appropriate.
> >
>
> For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> a. width
> b. height
> c. pixelformat
> d. plane_fmt
>     - sizeimage
>     - bytesperline
> e. num_planes
> Other fields are consider constant.
>
> So if the user space passes one pixel format with un-supported, we will
> apply the default format firstly and adjust width, height, sizeimage,
> and bytesperline. We will focus on validate width & height.
> Is it ok?

I'm not sure I understand your proposal, but let me describe the
proper behavior here:

if (pixelformat is invalid)
    pixelformat = some valid pixel format;

width = clamp(width, driver min, driver max);
height = clamp(height, driver min, driver max);

num_planes = 1;

calculate_sizeimage_and_bytesperline(fmt);

fill_in_the_remaining_constant_fields(fmt);

Does it make sense?

[snip]
> > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > +                           struct v4l2_format *f)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (cam_dev->streaming)
> > > +           return -EBUSY;
> >
> > I think this should rather be something like vb2_queue_is_busy(), which
> > would prevent format changes if buffers are allocated.
> >
>
> Since vb2_queue_is_busy is static function, would we paste its
> implementation in this function to check like this?
>
>         if (node->vdev.queue->owner &&
>                 node->vdev.queue->owner != file->private_data) {
>                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
>                 return -EBUSY;
>         }
>

Sorry, I mixed up the function name. That should've been vb2_is_busy().

[snip]
> > > +   /* Total pad numbers is video devices + one seninf pad */
> > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > +   unsigned int i;
> > > +   int ret;
> > > +
> > > +   ret = mtk_cam_media_register(dev,
> > > +                                &cam_dev->media_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > +           return ret;
> > > +   }
> > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> >
> > An info message should be useful to the user in some way. Printing kernel
> > pointers isn't useful. Something like "registered media0" could be useful to
> > let the user know which media device is associated with this driver if there
> > is more than one in the system.
> >
>
> Here is the new log info.
>
> dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
>

Let's fix the missing space and making a bit more readable:

dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);

>
> > > +
> > > +   /* Set up v4l2 device */
> > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > +           goto fail_v4l2_dev;
> >
> > Please call the labels after the cleanup step that needs to be done. It
> > makes it easier to spot any ordering errors.
> >
>
> Will fix in next patch.
>
> > > +   }
> > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> >
> > Same as above.
> >
>
> Ditto.
>
> dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
>

Perhaps just "Registered %s" to be consistent with the above media log?

[snip]
> > > +
> > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > +                                 struct v4l2_subdev *sd,
> > > +                                 struct v4l2_async_subdev *asd)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > +
> >
> > Should we somehow check that the entity we got is seninf indeed and there
> > was no mistake in DT?
> >
>
> How about to check the entity function of seninf device?
>
> if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>                 return -ENODEV;
> }
>
> If we need to check DT, may we need to implement this in parse_endpoint
> callback function of v4l2_async_notifier_parse_fwnode_endpoints?
>

Yes, checking the entity function is indeed the right way to do this.

[snip]
> > > +           .default_fmt_idx = 4,
> > > +           .max_buf_count = 10,
> >
> > Where does this number come from?
> >
>
> The default maximum VB2 buffer count is 32.
> In order to limit memory usage, we like to limit the maximum buffer
> counts in the driver layer. The maximum buffer count is estimated
> according to our camera MW.
>
> #define VB2_MAX_FRAME   (32)
>

Okay, thanks.

[snip]
> > > +   struct media_pad vdev_pad;
> > > +   struct vb2_queue vbq;
> > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > +   struct list_head pending_list;
> > > +   /* Used for vbq & vdev */
> >
> > It's already documented in the kerneldoc comment.
> >
>
> Fixed in next patch.
> Btw, if we remove this, we will got complain from checkpatch.pl script.
>

Oh really, that's weird. Okay, please keep it then, sorry for the confusion.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-23 10:21           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-23 10:21 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > +                                 struct media_request *new_req)
> > > +{
> > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +
> > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > +
> > > +   if (!cam_dev->streaming) {
> > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > +           spin_lock(&cam_dev->req_lock);
> > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > +           spin_unlock(&cam_dev->req_lock);
> > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Normal enqueue flow */
> > > +   if (new_req) {
> > > +           mtk_isp_req_enqueue(dev, new_req);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Flush all media requests wehen first stream on */
> > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > +           list_del(&req->list);
> > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > +   }
> > > +}
> >
> > This will have to be redone, as per the other suggestions, but generally one
> > would have a function that tries to queue as much as possible from a list to
> > the hardware and another function that adds a request to the list and calls
> > the first function.
> >
>
> We revised this function as below.
> First to check the en-queue conditions:
> a. stream on
> b. The composer buffers in SCP are 3, so we only could has 3 jobs
> at the same time.
>
>
> Second, try to en-queue the frames in the pending job if possible and
> move them into running job list if possible.
>
> The request has been inserted into pending job in mtk_cam_req_validate
> which is used to validate media_request.

Thanks for replying to each of the comments, that's very helpful.
Snipped out the parts that I agreed with.

Please note that req_validate is not supposed to change any driver
state. It's only supposed to validate the request. req_queue is the
right callback to insert the request into some internal driver
bookkeeping structures.

>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         struct list_head enqueue_job_list;
>         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
>         unsigned long flags;
>
>         if (!cam_dev->streaming ||
>             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {

Do we have a guarantee that cam_dev->running_job_count doesn't
decrement between the atomic_read() above and this line?

>                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
>                         buffer_cnt);
>                 return;
>         }
>
>         INIT_LIST_HEAD(&enqueue_job_list);
>
>         spin_lock(&cam_dev->pending_job_lock);
>         list_for_each_entry_safe(req, req_prev,
>                                  &cam_dev->pending_job_list, list) {
>                 list_del(&req->list);
>                 list_add_tail(&req->list, &enqueue_job_list);

What's the reason to use the second list? Could we just take one job
from pending_job_list, enqueue it and then iterate again?

>                 if (atomic_inc_return(&cam_dev->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;
>         }
>         spin_unlock(&cam_dev->pending_job_lock);
>
>         list_for_each_entry_safe(req, req_prev,
>                                  &enqueue_job_list, list) {
>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam_dev->running_job_list);
>                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
>

Do we have a guarantee that another thread doesn't run the same
function ending up calling mtk_isp_req_enqueue() with another request
before this one and thus making the order of running_job_list
incorrect?

>                 mtk_isp_req_enqueue(cam_dev, req);
>         }
> }
>
[snip]
> > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > +
> > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > +           stride = ALIGN(stride, 4);
> >
> > Is it expected that only the F10 format needs this alignment?
> >
>
> yes, if the pixel bits of image format is 10, the byte alignment of bpl
> should be 4. Otherwise, it is 8. We will revise this and add more
> comments.

That means that the B10 format also needs the extra alignment, as
opposed to what the original code did, right?

>
> /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
>         if (pixel_bits == 10)
>                 bpl = ALIGN(bpl, 4);
>         else
>                 bpl = ALIGN(bpl, 8);

SGTM, thanks.

[snip]
> > > +
> > > +static struct v4l2_subdev *
> > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > +   struct media_entity *entity;
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct v4l2_subdev *sensor;
> >
> > This variable would be unitialized if there is no streaming sensor. Was
> > there no compiler warning generated for this?
> >
>
> No, there is no compiler warning.
> But, we will assign sensor to NULL to avoid unnecessary compiler warning
> with different compiler options.
>

Thanks. It would be useful if you could check why the compiler you're
using doesn't show a warning here. We might be missing other
uninitialized variables.

[snip]
> > > +}
> > > +
> > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > +                               const struct media_pad *local,
> > > +                               const struct media_pad *remote, u32 flags)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > +   u32 pad = local->index;
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > +           __func__, pad, remote->index, flags);
> > > +
> > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> >
> > I assume this check is needed, because the pads with higher indexes are not
> > video nodes? If so, a comment would be helpful here.
> >
>
> Yes, we will new comment as below.
>
>         /*
>          * Check video nodes is enabled by link setup.
>          * The pad index of video node should be less than
>          * MTK_CAM_P1_TOTAL_NODES.
>          */
>         if (pad < MTK_CAM_P1_TOTAL_NODES)
>                 cam_dev->vdev_nodes[pad].enabled =
>                         !!(flags & MEDIA_LNK_FL_ENABLED);
>

Could we rephrase this a bit. The comment still doesn't explain why
the index should be less than the constant. Perhaps:

/*
 * The video nodes exposed by the driver have pads indexes
 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
 */

[snip]

> > > +
> > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > +           __func__,
> > > +           node->id,
> > > +           buf->vbb.request_fd,
> > > +           buf->vbb.vb2_buf.index);
> > > +
> > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > +   if (vb->vb2_queue->uses_requests)
> > > +           return;
> >
> > I'd suggest removing non-request support from this driver. Even if we end up
> > with a need to provide compatibility for non-request mode, then it should be
> > built on top of the requests mode, so that the driver itself doesn't have to
> > deal with two modes.
> >
>
> The purpose of non-request function in this driver is needed by
> our camera middle-ware design. It needs 3A statistics buffers before
> image buffers en-queue. So we need to en-queue 3A statistics with
> non-request mode in this driver. After MW got the 3A statistics data, it
> will en-queue the images, tuning buffer and other meta buffers with
> request mode. Based on this requirement, do you have any suggestion?
> For upstream driver, should we only consider request mode?
>

Where does that requirement come from? Why the timing of queuing of
the buffers to the driver is important?

[snip]
> > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > +                                  unsigned int count)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > +   int ret;
> > > +
> > > +   if (!node->enabled) {
> >
> > How is this synchronized with mtk_cam_media_link_setup()?
> >
>
> We will follow your suggestion and below is our proposal for this
> function.
>
> 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> stream-on.
> 2.a If yes, do the following steps
>     2.a-1 Call media_pipeline_start function to prevent the link
> configuration changes.
>     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> video nodes are enabled and save it into cam_dev->enabled_node_count.
>     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> user has called stream-on API
> 2.b jump step 3.
>
> 3. Use cam_dev->streamed_node_count to track how many video nodes are
> streamed by user space.
> 4. Check all enabled video nodes are streamed or not based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 5. If yes, call s_stream on for P1 sub-device
>
> Do you think it is reasonable?
>

That should work indeed.

[snip]
> > > +
> > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> >
> > Shouldn't we stop streaming first, so that the hardware operation is
> > cancelled and any buffers owned by the hardware are released?
> >
>
> For this function, below is the new code flow.
>
> 1. Check the first node to stream off based on
> cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> 2. If yes, call all s_stream off for P1 sub-device
> 3. Call mtk_cam_vb2_return_all_buffers for each node
> 4. Check the last node to stream off
> 5. If yes, call media_pipeline_stop to allow user space
> to perform link configuration changes, such as disable link.
>
> But, for step 5, is it too late for end user to disable link?
> For example, for first node, it has called stream off but
> can't call disable link until the last node is stream off?
>

I think that should be okay. From the userspace point of view, having
one of the video nodes streaming implies that the related subdevice
could be streaming as well. The links between the video nodes and the
subdevices don't have the DYNAMIC flag, so the userspace should expect
that it can't change any links connecting to the subdevice when the
subdevice could be streaming.

[snip]
> > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > +{
> > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > +
> > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > +                              dev->v4l2_dev.ctrl_handler);
> >
> > This would end up being called multiple times, once for each video node.
> > Instead, this should be called explicitly by the driver when it completed
> > the request - perhaps in the frame completion handler?
> >
> > With that, we probably wouldn't even need this callback.
> >
>
> First, if we don't implement this callback function, we will receive
> kernel warning as below.
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
>
> Second, this function is only be called in __vb2_queue_cancel function.
> Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> So could we just implement dummy empty function?
>
>  * @buf_request_complete: a buffer that was never queued to the driver
> but is
>  *                      associated with a queued request was canceled.
>  *                      The driver will have to mark associated objects in the
>  *                      request as completed; required if requests are
>  *                      supported.
>

Good catch, thanks.

Sounds like we may indeed need to implement this callback. In
particular, we may need to remove the request that the buffer was
associated with from the driver queue and return the other buffers
associated to it with an error state. This should be similar to
handling a request failure.
[snip]
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > +                              struct v4l2_fmtdesc *f)
> > > +{
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (f->index >= node->desc.num_fmts)
> > > +           return -EINVAL;
> > > +
> > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >
> > Is the set of formats available always the same regardless of the sensor
> > format?
> >
>
> Yes, ISP P1 HW output formats are always available without impact
> by sensor formats.
>
> > > +   f->flags = 0;
> >
> > We need f->description too.
> >
>
> For this description, do you suggest 1). we fill this field in this
> function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
>
> https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
>
> Basically, we prefer method 1.
>

That should be v4l_fill_fmtdesc(), as it already includes other
vendor-specific formats.

[snip]
> > > +
> > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > +           __func__,
> > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > +
> > > +   width = in_fmt->fmt.pix_mp.width;
> > > +   height = in_fmt->fmt.pix_mp.height;
> > > +
> > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > +   if (dev_fmt) {
> > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > +                                   &in_fmt->fmt.pix_mp,
> > > +                                   &dev_fmt->fmt.pix_mp,
> > > +                                   node->id);
> > > +   } else {
> > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > +                                        &node->desc, in_fmt);
> >
> > We shouldn't just load a default format. This function should validate all
> > the fields one by one and adjust them to something appropriate.
> >
>
> For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> a. width
> b. height
> c. pixelformat
> d. plane_fmt
>     - sizeimage
>     - bytesperline
> e. num_planes
> Other fields are consider constant.
>
> So if the user space passes one pixel format with un-supported, we will
> apply the default format firstly and adjust width, height, sizeimage,
> and bytesperline. We will focus on validate width & height.
> Is it ok?

I'm not sure I understand your proposal, but let me describe the
proper behavior here:

if (pixelformat is invalid)
    pixelformat = some valid pixel format;

width = clamp(width, driver min, driver max);
height = clamp(height, driver min, driver max);

num_planes = 1;

calculate_sizeimage_and_bytesperline(fmt);

fill_in_the_remaining_constant_fields(fmt);

Does it make sense?

[snip]
> > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > +                           struct v4l2_format *f)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > +
> > > +   if (cam_dev->streaming)
> > > +           return -EBUSY;
> >
> > I think this should rather be something like vb2_queue_is_busy(), which
> > would prevent format changes if buffers are allocated.
> >
>
> Since vb2_queue_is_busy is static function, would we paste its
> implementation in this function to check like this?
>
>         if (node->vdev.queue->owner &&
>                 node->vdev.queue->owner != file->private_data) {
>                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
>                 return -EBUSY;
>         }
>

Sorry, I mixed up the function name. That should've been vb2_is_busy().

[snip]
> > > +   /* Total pad numbers is video devices + one seninf pad */
> > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > +   unsigned int i;
> > > +   int ret;
> > > +
> > > +   ret = mtk_cam_media_register(dev,
> > > +                                &cam_dev->media_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > +           return ret;
> > > +   }
> > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> >
> > An info message should be useful to the user in some way. Printing kernel
> > pointers isn't useful. Something like "registered media0" could be useful to
> > let the user know which media device is associated with this driver if there
> > is more than one in the system.
> >
>
> Here is the new log info.
>
> dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
>

Let's fix the missing space and making a bit more readable:

dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);

>
> > > +
> > > +   /* Set up v4l2 device */
> > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > +   if (ret) {
> > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > +           goto fail_v4l2_dev;
> >
> > Please call the labels after the cleanup step that needs to be done. It
> > makes it easier to spot any ordering errors.
> >
>
> Will fix in next patch.
>
> > > +   }
> > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> >
> > Same as above.
> >
>
> Ditto.
>
> dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
>

Perhaps just "Registered %s" to be consistent with the above media log?

[snip]
> > > +
> > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > +                                 struct v4l2_subdev *sd,
> > > +                                 struct v4l2_async_subdev *asd)
> > > +{
> > > +   struct mtk_cam_dev *cam_dev =
> > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > +
> >
> > Should we somehow check that the entity we got is seninf indeed and there
> > was no mistake in DT?
> >
>
> How about to check the entity function of seninf device?
>
> if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>                 return -ENODEV;
> }
>
> If we need to check DT, may we need to implement this in parse_endpoint
> callback function of v4l2_async_notifier_parse_fwnode_endpoints?
>

Yes, checking the entity function is indeed the right way to do this.

[snip]
> > > +           .default_fmt_idx = 4,
> > > +           .max_buf_count = 10,
> >
> > Where does this number come from?
> >
>
> The default maximum VB2 buffer count is 32.
> In order to limit memory usage, we like to limit the maximum buffer
> counts in the driver layer. The maximum buffer count is estimated
> according to our camera MW.
>
> #define VB2_MAX_FRAME   (32)
>

Okay, thanks.

[snip]
> > > +   struct media_pad vdev_pad;
> > > +   struct vb2_queue vbq;
> > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > +   struct list_head pending_list;
> > > +   /* Used for vbq & vdev */
> >
> > It's already documented in the kerneldoc comment.
> >
>
> Fixed in next patch.
> Btw, if we remove this, we will got complain from checkpatch.pl script.
>

Oh really, that's weird. Okay, please keep it then, sorry for the confusion.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-23 10:21           ` Tomasz Figa
  (?)
@ 2019-07-24  4:31             ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-24  4:31 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab, list

Hi, Tomasz:

On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > +                                 struct media_request *new_req)
> > > > +{
> > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +
> > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > +
> > > > +   if (!cam_dev->streaming) {
> > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > +           spin_lock(&cam_dev->req_lock);
> > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > +           spin_unlock(&cam_dev->req_lock);
> > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Normal enqueue flow */
> > > > +   if (new_req) {
> > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Flush all media requests wehen first stream on */
> > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > +           list_del(&req->list);
> > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > +   }
> > > > +}
> > >
> > > This will have to be redone, as per the other suggestions, but generally one
> > > would have a function that tries to queue as much as possible from a list to
> > > the hardware and another function that adds a request to the list and calls
> > > the first function.
> > >
> >
> > We revised this function as below.
> > First to check the en-queue conditions:
> > a. stream on
> > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > at the same time.
> >
> >
> > Second, try to en-queue the frames in the pending job if possible and
> > move them into running job list if possible.
> >
> > The request has been inserted into pending job in mtk_cam_req_validate
> > which is used to validate media_request.
> 
> Thanks for replying to each of the comments, that's very helpful.
> Snipped out the parts that I agreed with.
> 
> Please note that req_validate is not supposed to change any driver
> state. It's only supposed to validate the request. req_queue is the
> right callback to insert the request into some internal driver
> bookkeeping structures.
> 

Yes, in req_validate function, we don't change any driver state.
Below is the function's implementation.

a. Call vb2_request_validate(req) to verify media request.
b. Update the buffer internal structure buffer.
c. Insert the request into pending_job_list to prepare en-queue.

static int mtk_cam_req_validate(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);
	struct media_request_object *req_obj;
	unsigned long flags;
	int ret;

	/* run buffer prepare function to initialize buffer DMA address */
	ret = vb2_request_validate(req);
	if (ret) {
		dev_err(cam->dev, "vb2_request_validate failed:%d\n", ret);
		return ret;
	}

	/* update frame_params */
	list_for_each_entry(req_obj, &req->objects, list) {
		struct vb2_buffer *vb;
		struct mtk_cam_dev_buffer *buf;

		if (!vb2_request_object_is_buffer(req_obj))
			continue;

		vb = container_of(req_obj, struct vb2_buffer, req_obj);
		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
		cam_req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
		cam_req->frame_params.dma_bufs[buf->node_id].scp_addr =
			buf->scp_addr;
	}
	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	return 0;
}

> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         struct list_head enqueue_job_list;
> >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> >         unsigned long flags;
> >
> >         if (!cam_dev->streaming ||
> >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> Do we have a guarantee that cam_dev->running_job_count doesn't
> decrement between the atomic_read() above and this line?
> 

Ok, we will use cam->pending_job_lock to protect
cam_dev->running_job_count access. Below is the revised version.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "jobs are full\n");
		spin_unlock_irqrestore(&cam->pending_job_lock, flags);
		return;
	}
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
}

> >                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
> >                         buffer_cnt);
> >                 return;
> >         }
> >
> >         INIT_LIST_HEAD(&enqueue_job_list);
> >
> >         spin_lock(&cam_dev->pending_job_lock);
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &cam_dev->pending_job_list, list) {
> >                 list_del(&req->list);
> >                 list_add_tail(&req->list, &enqueue_job_list);
> 
> What's the reason to use the second list? Could we just take one job
> from pending_job_list, enqueue it and then iterate again?
> 

Yes, we could simply the code block to remove enqueue_job_list.

> >                 if (atomic_inc_return(&cam_dev->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> >         }
> >         spin_unlock(&cam_dev->pending_job_lock);
> >
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &enqueue_job_list, list) {
> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam_dev->running_job_list);
> >                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
> >
> 
> Do we have a guarantee that another thread doesn't run the same
> function ending up calling mtk_isp_req_enqueue() with another request
> before this one and thus making the order of running_job_list
> incorrect?
> 

In the new implementation, we use cam->pending_job_lock to protect this
scenario.

> >                 mtk_isp_req_enqueue(cam_dev, req);
> >         }
> > }
> >
> [snip]
> > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > +
> > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > +           stride = ALIGN(stride, 4);
> > >
> > > Is it expected that only the F10 format needs this alignment?
> > >
> >
> > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > should be 4. Otherwise, it is 8. We will revise this and add more
> > comments.
> 
> That means that the B10 format also needs the extra alignment, as
> opposed to what the original code did, right?
> 

Sorry for short code snippet.
This alignment checking is only applied to F10, no B10.
If you like to check the full function, you could check this in this
link[1].

static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
node_id,
			     struct v4l2_pix_format_mplane *mp)
{
	unsigned int bpl, ppl;
	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
	unsigned int width = mp->width;

	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
		/* bayer encoding format & 2 bytes alignment */
		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
		/*
		 * The FULL-G encoding format
		 * 1 G component per pixel
		 * 1 R component per 4 pixel
		 * 1 B component per 4 pixel
		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
		 */
		ppl = DIV_ROUND_UP(width * 6, 4);
		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);

		/* 4 bytes alignment for 10 bit & others are 8 bytes */
		if (pixel_bits == 10)
			bpl = ALIGN(bpl, 4);
		else
			bpl = ALIGN(bpl, 8);
	} 

[1]
https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303

> >
> > /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
> >         if (pixel_bits == 10)
> >                 bpl = ALIGN(bpl, 4);
> >         else
> >                 bpl = ALIGN(bpl, 8);
> 
> SGTM, thanks.
> 
> [snip]

Thanks for your review.

> > > > +
> > > > +static struct v4l2_subdev *
> > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > +   struct media_entity *entity;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct v4l2_subdev *sensor;
> > >
> > > This variable would be unitialized if there is no streaming sensor. Was
> > > there no compiler warning generated for this?
> > >
> >
> > No, there is no compiler warning.
> > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > with different compiler options.
> >
> 
> Thanks. It would be useful if you could check why the compiler you're
> using doesn't show a warning here. We might be missing other
> uninitialized variables.
> 

We will feedback to your project team to check the possible reason about
compiler warning issue.

> [snip]
> > > > +}
> > > > +
> > > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > > +                               const struct media_pad *local,
> > > > +                               const struct media_pad *remote, u32 flags)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > > +   u32 pad = local->index;
> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > > +           __func__, pad, remote->index, flags);
> > > > +
> > > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> > >
> > > I assume this check is needed, because the pads with higher indexes are not
> > > video nodes? If so, a comment would be helpful here.
> > >
> >
> > Yes, we will new comment as below.
> >
> >         /*
> >          * Check video nodes is enabled by link setup.
> >          * The pad index of video node should be less than
> >          * MTK_CAM_P1_TOTAL_NODES.
> >          */
> >         if (pad < MTK_CAM_P1_TOTAL_NODES)
> >                 cam_dev->vdev_nodes[pad].enabled =
> >                         !!(flags & MEDIA_LNK_FL_ENABLED);
> >
> 
> Could we rephrase this a bit. The comment still doesn't explain why
> the index should be less than the constant. Perhaps:
> 
> /*
>  * The video nodes exposed by the driver have pads indexes
>  * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
>  */
> 
> [snip]
> 

Thanks for your suggestion.
We will update this.

> > > > +
> > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > +           __func__,
> > > > +           node->id,
> > > > +           buf->vbb.request_fd,
> > > > +           buf->vbb.vb2_buf.index);
> > > > +
> > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > +   if (vb->vb2_queue->uses_requests)
> > > > +           return;
> > >
> > > I'd suggest removing non-request support from this driver. Even if we end up
> > > with a need to provide compatibility for non-request mode, then it should be
> > > built on top of the requests mode, so that the driver itself doesn't have to
> > > deal with two modes.
> > >
> >
> > The purpose of non-request function in this driver is needed by
> > our camera middle-ware design. It needs 3A statistics buffers before
> > image buffers en-queue. So we need to en-queue 3A statistics with
> > non-request mode in this driver. After MW got the 3A statistics data, it
> > will en-queue the images, tuning buffer and other meta buffers with
> > request mode. Based on this requirement, do you have any suggestion?
> > For upstream driver, should we only consider request mode?
> >
> 
> Where does that requirement come from? Why the timing of queuing of
> the buffers to the driver is important?
> 
> [snip]

Basically, this requirement comes from our internal camera
middle-ware/3A hal in user space. Since this is not generic requirement,
we will follow your original suggestion to keep the request mode only
and remove other non-request design in other files. For upstream driver,
it should support request mode only.

> > > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > > +                                  unsigned int count)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > > +   int ret;
> > > > +
> > > > +   if (!node->enabled) {
> > >
> > > How is this synchronized with mtk_cam_media_link_setup()?
> > >
> >
> > We will follow your suggestion and below is our proposal for this
> > function.
> >
> > 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> > stream-on.
> > 2.a If yes, do the following steps
> >     2.a-1 Call media_pipeline_start function to prevent the link
> > configuration changes.
> >     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> > video nodes are enabled and save it into cam_dev->enabled_node_count.
> >     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> > user has called stream-on API
> > 2.b jump step 3.
> >
> > 3. Use cam_dev->streamed_node_count to track how many video nodes are
> > streamed by user space.
> > 4. Check all enabled video nodes are streamed or not based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 5. If yes, call s_stream on for P1 sub-device
> >
> > Do you think it is reasonable?
> >
> 
> That should work indeed.
> 
> [snip]

Ok, thanks for your confirmation.

> > > > +
> > > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > >
> > > Shouldn't we stop streaming first, so that the hardware operation is
> > > cancelled and any buffers owned by the hardware are released?
> > >
> >
> > For this function, below is the new code flow.
> >
> > 1. Check the first node to stream off based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 2. If yes, call all s_stream off for P1 sub-device
> > 3. Call mtk_cam_vb2_return_all_buffers for each node
> > 4. Check the last node to stream off
> > 5. If yes, call media_pipeline_stop to allow user space
> > to perform link configuration changes, such as disable link.
> >
> > But, for step 5, is it too late for end user to disable link?
> > For example, for first node, it has called stream off but
> > can't call disable link until the last node is stream off?
> >
> 
> I think that should be okay. From the userspace point of view, having
> one of the video nodes streaming implies that the related subdevice
> could be streaming as well. The links between the video nodes and the
> subdevices don't have the DYNAMIC flag, so the userspace should expect
> that it can't change any links connecting to the subdevice when the
> subdevice could be streaming.
> 

Ok, got your point. We will keep this design.

> [snip]
> > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > +{
> > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > +
> > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > +                              dev->v4l2_dev.ctrl_handler);
> > >
> > > This would end up being called multiple times, once for each video node.
> > > Instead, this should be called explicitly by the driver when it completed
> > > the request - perhaps in the frame completion handler?
> > >
> > > With that, we probably wouldn't even need this callback.
> > >
> >
> > First, if we don't implement this callback function, we will receive
> > kernel warning as below.
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> >
> > Second, this function is only be called in __vb2_queue_cancel function.
> > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > So could we just implement dummy empty function?
> >
> >  * @buf_request_complete: a buffer that was never queued to the driver
> > but is
> >  *                      associated with a queued request was canceled.
> >  *                      The driver will have to mark associated objects in the
> >  *                      request as completed; required if requests are
> >  *                      supported.
> >
> 
> Good catch, thanks.
> 
> Sounds like we may indeed need to implement this callback. In
> particular, we may need to remove the request that the buffer was
> associated with from the driver queue and return the other buffers
> associated to it with an error state. This should be similar to
> handling a request failure.
> [snip]

Before calling this callback function, the VB2's stop_streaming has been
called. Normally, we will return the buffers belonged to this vb2 queu
with error state. On other hand, only if the state of request is
MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
__vb2_queue_cancel function. It hints this media request has been
validated and inserted into our driver's pending_job_list or
running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
requests from driver's list when streaming is off. Since we have no
v4l2_ctrl, do we need to do the above things which is already handled in
mtk_cam_vb2_stop_streaming function? Maybe is this callback function
only designed for v4l2_ctrl_request_complete usage?

static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	dev_dbg(cam->dev, "%s\n", __func__);

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
}

static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
{
	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
	struct device *dev = cam->dev;

	dev_dbg(dev, "%s node:%d count info:%d", __func__,
		node->id, atomic_read(&cam->stream_count));

	mutex_lock(&cam->op_lock);
	if (atomic_read(&cam->stream_count) == cam->enabled_count)
		if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
			dev_err(dev, "failed to stop streaming\n");

	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);

	/* Check the first node to stream-off */
	if (!atomic_dec_and_test(&cam->stream_count)) {
		mutex_unlock(&cam->op_lock);
		return;
	}
	mutex_unlock(&cam->op_lock);

	mtk_cam_dev_req_cleanup(cam);
	media_pipeline_stop(&node->vdev.entity);
}

> > > > +
> > > > +   return 0;
> > > > +}
> > > > +
> > > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > > +                              struct v4l2_fmtdesc *f)
> > > > +{
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (f->index >= node->desc.num_fmts)
> > > > +           return -EINVAL;
> > > > +
> > > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > >
> > > Is the set of formats available always the same regardless of the sensor
> > > format?
> > >
> >
> > Yes, ISP P1 HW output formats are always available without impact
> > by sensor formats.
> >
> > > > +   f->flags = 0;
> > >
> > > We need f->description too.
> > >
> >
> > For this description, do you suggest 1). we fill this field in this
> > function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
> >
> > Basically, we prefer method 1.
> >
> 
> That should be v4l_fill_fmtdesc(), as it already includes other
> vendor-specific formats.
> 
> [snip]


Ok, got it. We will follow your suggestion.

> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > > +           __func__,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > > +
> > > > +   width = in_fmt->fmt.pix_mp.width;
> > > > +   height = in_fmt->fmt.pix_mp.height;
> > > > +
> > > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > > +   if (dev_fmt) {
> > > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > > +                                   &in_fmt->fmt.pix_mp,
> > > > +                                   &dev_fmt->fmt.pix_mp,
> > > > +                                   node->id);
> > > > +   } else {
> > > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > > +                                        &node->desc, in_fmt);
> > >
> > > We shouldn't just load a default format. This function should validate all
> > > the fields one by one and adjust them to something appropriate.
> > >
> >
> > For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> > a. width
> > b. height
> > c. pixelformat
> > d. plane_fmt
> >     - sizeimage
> >     - bytesperline
> > e. num_planes
> > Other fields are consider constant.
> >
> > So if the user space passes one pixel format with un-supported, we will
> > apply the default format firstly and adjust width, height, sizeimage,
> > and bytesperline. We will focus on validate width & height.
> > Is it ok?
> 
> I'm not sure I understand your proposal, but let me describe the
> proper behavior here:
> 
> if (pixelformat is invalid)
>     pixelformat = some valid pixel format;
> 
> width = clamp(width, driver min, driver max);
> height = clamp(height, driver min, driver max);
> 
> num_planes = 1;
> 
> calculate_sizeimage_and_bytesperline(fmt);
> 
> fill_in_the_remaining_constant_fields(fmt);
> 
> Does it make sense?
> 
> [snip]

Yes, here is our new version.

static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
				  struct v4l2_format *f)
{
	struct mtk_cam_dev *cam = video_drvdata(file);
	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
	struct device *dev = cam->dev;
	const struct v4l2_format *dev_fmt;
	struct v4l2_format try_fmt;

	dev_dbg(dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
		__func__,
		(f->fmt.pix_mp.pixelformat & 0xFF),
		(f->fmt.pix_mp.pixelformat >> 8) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 16) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 24) & 0xFF,
		f->fmt.pix_mp.width, f->fmt.pix_mp.height);

	memset(&try_fmt, 0, sizeof(try_fmt));
	try_fmt.type = f->type;

	/* Validate pixelformat */
	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
	if (!dev_fmt) {
		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
	}
	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;

	/* Validate image width & height range */
	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);

	/* 4 bytes alignment for width */
	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);

	/* bytesperline & sizeimage calculation */
	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);

	/* Constant format fields */
	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
	try_fmt.fmt.pix_mp.num_planes = 1;
	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;

	*f = try_fmt;

	return 0;
}

> > > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > > +                           struct v4l2_format *f)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (cam_dev->streaming)
> > > > +           return -EBUSY;
> > >
> > > I think this should rather be something like vb2_queue_is_busy(), which
> > > would prevent format changes if buffers are allocated.
> > >
> >
> > Since vb2_queue_is_busy is static function, would we paste its
> > implementation in this function to check like this?
> >
> >         if (node->vdev.queue->owner &&
> >                 node->vdev.queue->owner != file->private_data) {
> >                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
> >                 return -EBUSY;
> >         }
> >
> 
> Sorry, I mixed up the function name. That should've been vb2_is_busy().
> 
> [snip]

Got it. Thanks for your suggestion.

> > > > +   /* Total pad numbers is video devices + one seninf pad */
> > > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > > +   unsigned int i;
> > > > +   int ret;
> > > > +
> > > > +   ret = mtk_cam_media_register(dev,
> > > > +                                &cam_dev->media_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > > +           return ret;
> > > > +   }
> > > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> > >
> > > An info message should be useful to the user in some way. Printing kernel
> > > pointers isn't useful. Something like "registered media0" could be useful to
> > > let the user know which media device is associated with this driver if there
> > > is more than one in the system.
> > >
> >
> > Here is the new log info.
> >
> > dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
> >
> 
> Let's fix the missing space and making a bit more readable:
> 
> dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);
> 

Ok, we will apply this change.

> >
> > > > +
> > > > +   /* Set up v4l2 device */
> > > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > > +           goto fail_v4l2_dev;
> > >
> > > Please call the labels after the cleanup step that needs to be done. It
> > > makes it easier to spot any ordering errors.
> > >
> >
> > Will fix in next patch.
> >
> > > > +   }
> > > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> > >
> > > Same as above.
> > >
> >
> > Ditto.
> >
> > dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
> >
> 
> Perhaps just "Registered %s" to be consistent with the above media log?
> 
> [snip]

Ditto.

> > > > +
> > > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > > +                                 struct v4l2_subdev *sd,
> > > > +                                 struct v4l2_async_subdev *asd)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > > +
> > >
> > > Should we somehow check that the entity we got is seninf indeed and there
> > > was no mistake in DT?
> > >
> >
> > How about to check the entity function of seninf device?
> >
> > if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >                 return -ENODEV;
> > }
> >
> > If we need to check DT, may we need to implement this in parse_endpoint
> > callback function of v4l2_async_notifier_parse_fwnode_endpoints?
> >
> 
> Yes, checking the entity function is indeed the right way to do this.
> 
> [snip]

Thanks for your confirm.

> > > > +           .default_fmt_idx = 4,
> > > > +           .max_buf_count = 10,
> > >
> > > Where does this number come from?
> > >
> >
> > The default maximum VB2 buffer count is 32.
> > In order to limit memory usage, we like to limit the maximum buffer
> > counts in the driver layer. The maximum buffer count is estimated
> > according to our camera MW.
> >
> > #define VB2_MAX_FRAME   (32)
> >
> 
> Okay, thanks.
> 
> [snip]
> > > > +   struct media_pad vdev_pad;
> > > > +   struct vb2_queue vbq;
> > > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > > +   struct list_head pending_list;
> > > > +   /* Used for vbq & vdev */
> > >
> > > It's already documented in the kerneldoc comment.
> > >
> >
> > Fixed in next patch.
> > Btw, if we remove this, we will got complain from checkpatch.pl script.
> >
> 
> Oh really, that's weird. Okay, please keep it then, sorry for the confusion.
> 
> Best regards,
> Tomasz

Ok, thanks for your understanding.
We will rollback this change to avoid checkpatch's complains.


Best regards,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-24  4:31             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-24  4:31 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi, Tomasz:

On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > +                                 struct media_request *new_req)
> > > > +{
> > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +
> > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > +
> > > > +   if (!cam_dev->streaming) {
> > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > +           spin_lock(&cam_dev->req_lock);
> > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > +           spin_unlock(&cam_dev->req_lock);
> > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Normal enqueue flow */
> > > > +   if (new_req) {
> > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Flush all media requests wehen first stream on */
> > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > +           list_del(&req->list);
> > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > +   }
> > > > +}
> > >
> > > This will have to be redone, as per the other suggestions, but generally one
> > > would have a function that tries to queue as much as possible from a list to
> > > the hardware and another function that adds a request to the list and calls
> > > the first function.
> > >
> >
> > We revised this function as below.
> > First to check the en-queue conditions:
> > a. stream on
> > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > at the same time.
> >
> >
> > Second, try to en-queue the frames in the pending job if possible and
> > move them into running job list if possible.
> >
> > The request has been inserted into pending job in mtk_cam_req_validate
> > which is used to validate media_request.
> 
> Thanks for replying to each of the comments, that's very helpful.
> Snipped out the parts that I agreed with.
> 
> Please note that req_validate is not supposed to change any driver
> state. It's only supposed to validate the request. req_queue is the
> right callback to insert the request into some internal driver
> bookkeeping structures.
> 

Yes, in req_validate function, we don't change any driver state.
Below is the function's implementation.

a. Call vb2_request_validate(req) to verify media request.
b. Update the buffer internal structure buffer.
c. Insert the request into pending_job_list to prepare en-queue.

static int mtk_cam_req_validate(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);
	struct media_request_object *req_obj;
	unsigned long flags;
	int ret;

	/* run buffer prepare function to initialize buffer DMA address */
	ret = vb2_request_validate(req);
	if (ret) {
		dev_err(cam->dev, "vb2_request_validate failed:%d\n", ret);
		return ret;
	}

	/* update frame_params */
	list_for_each_entry(req_obj, &req->objects, list) {
		struct vb2_buffer *vb;
		struct mtk_cam_dev_buffer *buf;

		if (!vb2_request_object_is_buffer(req_obj))
			continue;

		vb = container_of(req_obj, struct vb2_buffer, req_obj);
		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
		cam_req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
		cam_req->frame_params.dma_bufs[buf->node_id].scp_addr =
			buf->scp_addr;
	}
	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	return 0;
}

> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         struct list_head enqueue_job_list;
> >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> >         unsigned long flags;
> >
> >         if (!cam_dev->streaming ||
> >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> Do we have a guarantee that cam_dev->running_job_count doesn't
> decrement between the atomic_read() above and this line?
> 

Ok, we will use cam->pending_job_lock to protect
cam_dev->running_job_count access. Below is the revised version.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "jobs are full\n");
		spin_unlock_irqrestore(&cam->pending_job_lock, flags);
		return;
	}
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
}

> >                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
> >                         buffer_cnt);
> >                 return;
> >         }
> >
> >         INIT_LIST_HEAD(&enqueue_job_list);
> >
> >         spin_lock(&cam_dev->pending_job_lock);
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &cam_dev->pending_job_list, list) {
> >                 list_del(&req->list);
> >                 list_add_tail(&req->list, &enqueue_job_list);
> 
> What's the reason to use the second list? Could we just take one job
> from pending_job_list, enqueue it and then iterate again?
> 

Yes, we could simply the code block to remove enqueue_job_list.

> >                 if (atomic_inc_return(&cam_dev->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> >         }
> >         spin_unlock(&cam_dev->pending_job_lock);
> >
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &enqueue_job_list, list) {
> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam_dev->running_job_list);
> >                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
> >
> 
> Do we have a guarantee that another thread doesn't run the same
> function ending up calling mtk_isp_req_enqueue() with another request
> before this one and thus making the order of running_job_list
> incorrect?
> 

In the new implementation, we use cam->pending_job_lock to protect this
scenario.

> >                 mtk_isp_req_enqueue(cam_dev, req);
> >         }
> > }
> >
> [snip]
> > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > +
> > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > +           stride = ALIGN(stride, 4);
> > >
> > > Is it expected that only the F10 format needs this alignment?
> > >
> >
> > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > should be 4. Otherwise, it is 8. We will revise this and add more
> > comments.
> 
> That means that the B10 format also needs the extra alignment, as
> opposed to what the original code did, right?
> 

Sorry for short code snippet.
This alignment checking is only applied to F10, no B10.
If you like to check the full function, you could check this in this
link[1].

static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
node_id,
			     struct v4l2_pix_format_mplane *mp)
{
	unsigned int bpl, ppl;
	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
	unsigned int width = mp->width;

	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
		/* bayer encoding format & 2 bytes alignment */
		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
		/*
		 * The FULL-G encoding format
		 * 1 G component per pixel
		 * 1 R component per 4 pixel
		 * 1 B component per 4 pixel
		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
		 */
		ppl = DIV_ROUND_UP(width * 6, 4);
		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);

		/* 4 bytes alignment for 10 bit & others are 8 bytes */
		if (pixel_bits == 10)
			bpl = ALIGN(bpl, 4);
		else
			bpl = ALIGN(bpl, 8);
	} 

[1]
https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303

> >
> > /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
> >         if (pixel_bits == 10)
> >                 bpl = ALIGN(bpl, 4);
> >         else
> >                 bpl = ALIGN(bpl, 8);
> 
> SGTM, thanks.
> 
> [snip]

Thanks for your review.

> > > > +
> > > > +static struct v4l2_subdev *
> > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > +   struct media_entity *entity;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct v4l2_subdev *sensor;
> > >
> > > This variable would be unitialized if there is no streaming sensor. Was
> > > there no compiler warning generated for this?
> > >
> >
> > No, there is no compiler warning.
> > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > with different compiler options.
> >
> 
> Thanks. It would be useful if you could check why the compiler you're
> using doesn't show a warning here. We might be missing other
> uninitialized variables.
> 

We will feedback to your project team to check the possible reason about
compiler warning issue.

> [snip]
> > > > +}
> > > > +
> > > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > > +                               const struct media_pad *local,
> > > > +                               const struct media_pad *remote, u32 flags)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > > +   u32 pad = local->index;
> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > > +           __func__, pad, remote->index, flags);
> > > > +
> > > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> > >
> > > I assume this check is needed, because the pads with higher indexes are not
> > > video nodes? If so, a comment would be helpful here.
> > >
> >
> > Yes, we will new comment as below.
> >
> >         /*
> >          * Check video nodes is enabled by link setup.
> >          * The pad index of video node should be less than
> >          * MTK_CAM_P1_TOTAL_NODES.
> >          */
> >         if (pad < MTK_CAM_P1_TOTAL_NODES)
> >                 cam_dev->vdev_nodes[pad].enabled =
> >                         !!(flags & MEDIA_LNK_FL_ENABLED);
> >
> 
> Could we rephrase this a bit. The comment still doesn't explain why
> the index should be less than the constant. Perhaps:
> 
> /*
>  * The video nodes exposed by the driver have pads indexes
>  * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
>  */
> 
> [snip]
> 

Thanks for your suggestion.
We will update this.

> > > > +
> > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > +           __func__,
> > > > +           node->id,
> > > > +           buf->vbb.request_fd,
> > > > +           buf->vbb.vb2_buf.index);
> > > > +
> > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > +   if (vb->vb2_queue->uses_requests)
> > > > +           return;
> > >
> > > I'd suggest removing non-request support from this driver. Even if we end up
> > > with a need to provide compatibility for non-request mode, then it should be
> > > built on top of the requests mode, so that the driver itself doesn't have to
> > > deal with two modes.
> > >
> >
> > The purpose of non-request function in this driver is needed by
> > our camera middle-ware design. It needs 3A statistics buffers before
> > image buffers en-queue. So we need to en-queue 3A statistics with
> > non-request mode in this driver. After MW got the 3A statistics data, it
> > will en-queue the images, tuning buffer and other meta buffers with
> > request mode. Based on this requirement, do you have any suggestion?
> > For upstream driver, should we only consider request mode?
> >
> 
> Where does that requirement come from? Why the timing of queuing of
> the buffers to the driver is important?
> 
> [snip]

Basically, this requirement comes from our internal camera
middle-ware/3A hal in user space. Since this is not generic requirement,
we will follow your original suggestion to keep the request mode only
and remove other non-request design in other files. For upstream driver,
it should support request mode only.

> > > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > > +                                  unsigned int count)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > > +   int ret;
> > > > +
> > > > +   if (!node->enabled) {
> > >
> > > How is this synchronized with mtk_cam_media_link_setup()?
> > >
> >
> > We will follow your suggestion and below is our proposal for this
> > function.
> >
> > 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> > stream-on.
> > 2.a If yes, do the following steps
> >     2.a-1 Call media_pipeline_start function to prevent the link
> > configuration changes.
> >     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> > video nodes are enabled and save it into cam_dev->enabled_node_count.
> >     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> > user has called stream-on API
> > 2.b jump step 3.
> >
> > 3. Use cam_dev->streamed_node_count to track how many video nodes are
> > streamed by user space.
> > 4. Check all enabled video nodes are streamed or not based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 5. If yes, call s_stream on for P1 sub-device
> >
> > Do you think it is reasonable?
> >
> 
> That should work indeed.
> 
> [snip]

Ok, thanks for your confirmation.

> > > > +
> > > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > >
> > > Shouldn't we stop streaming first, so that the hardware operation is
> > > cancelled and any buffers owned by the hardware are released?
> > >
> >
> > For this function, below is the new code flow.
> >
> > 1. Check the first node to stream off based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 2. If yes, call all s_stream off for P1 sub-device
> > 3. Call mtk_cam_vb2_return_all_buffers for each node
> > 4. Check the last node to stream off
> > 5. If yes, call media_pipeline_stop to allow user space
> > to perform link configuration changes, such as disable link.
> >
> > But, for step 5, is it too late for end user to disable link?
> > For example, for first node, it has called stream off but
> > can't call disable link until the last node is stream off?
> >
> 
> I think that should be okay. From the userspace point of view, having
> one of the video nodes streaming implies that the related subdevice
> could be streaming as well. The links between the video nodes and the
> subdevices don't have the DYNAMIC flag, so the userspace should expect
> that it can't change any links connecting to the subdevice when the
> subdevice could be streaming.
> 

Ok, got your point. We will keep this design.

> [snip]
> > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > +{
> > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > +
> > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > +                              dev->v4l2_dev.ctrl_handler);
> > >
> > > This would end up being called multiple times, once for each video node.
> > > Instead, this should be called explicitly by the driver when it completed
> > > the request - perhaps in the frame completion handler?
> > >
> > > With that, we probably wouldn't even need this callback.
> > >
> >
> > First, if we don't implement this callback function, we will receive
> > kernel warning as below.
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> >
> > Second, this function is only be called in __vb2_queue_cancel function.
> > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > So could we just implement dummy empty function?
> >
> >  * @buf_request_complete: a buffer that was never queued to the driver
> > but is
> >  *                      associated with a queued request was canceled.
> >  *                      The driver will have to mark associated objects in the
> >  *                      request as completed; required if requests are
> >  *                      supported.
> >
> 
> Good catch, thanks.
> 
> Sounds like we may indeed need to implement this callback. In
> particular, we may need to remove the request that the buffer was
> associated with from the driver queue and return the other buffers
> associated to it with an error state. This should be similar to
> handling a request failure.
> [snip]

Before calling this callback function, the VB2's stop_streaming has been
called. Normally, we will return the buffers belonged to this vb2 queu
with error state. On other hand, only if the state of request is
MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
__vb2_queue_cancel function. It hints this media request has been
validated and inserted into our driver's pending_job_list or
running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
requests from driver's list when streaming is off. Since we have no
v4l2_ctrl, do we need to do the above things which is already handled in
mtk_cam_vb2_stop_streaming function? Maybe is this callback function
only designed for v4l2_ctrl_request_complete usage?

static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	dev_dbg(cam->dev, "%s\n", __func__);

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
}

static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
{
	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
	struct device *dev = cam->dev;

	dev_dbg(dev, "%s node:%d count info:%d", __func__,
		node->id, atomic_read(&cam->stream_count));

	mutex_lock(&cam->op_lock);
	if (atomic_read(&cam->stream_count) == cam->enabled_count)
		if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
			dev_err(dev, "failed to stop streaming\n");

	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);

	/* Check the first node to stream-off */
	if (!atomic_dec_and_test(&cam->stream_count)) {
		mutex_unlock(&cam->op_lock);
		return;
	}
	mutex_unlock(&cam->op_lock);

	mtk_cam_dev_req_cleanup(cam);
	media_pipeline_stop(&node->vdev.entity);
}

> > > > +
> > > > +   return 0;
> > > > +}
> > > > +
> > > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > > +                              struct v4l2_fmtdesc *f)
> > > > +{
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (f->index >= node->desc.num_fmts)
> > > > +           return -EINVAL;
> > > > +
> > > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > >
> > > Is the set of formats available always the same regardless of the sensor
> > > format?
> > >
> >
> > Yes, ISP P1 HW output formats are always available without impact
> > by sensor formats.
> >
> > > > +   f->flags = 0;
> > >
> > > We need f->description too.
> > >
> >
> > For this description, do you suggest 1). we fill this field in this
> > function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
> >
> > Basically, we prefer method 1.
> >
> 
> That should be v4l_fill_fmtdesc(), as it already includes other
> vendor-specific formats.
> 
> [snip]


Ok, got it. We will follow your suggestion.

> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > > +           __func__,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > > +
> > > > +   width = in_fmt->fmt.pix_mp.width;
> > > > +   height = in_fmt->fmt.pix_mp.height;
> > > > +
> > > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > > +   if (dev_fmt) {
> > > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > > +                                   &in_fmt->fmt.pix_mp,
> > > > +                                   &dev_fmt->fmt.pix_mp,
> > > > +                                   node->id);
> > > > +   } else {
> > > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > > +                                        &node->desc, in_fmt);
> > >
> > > We shouldn't just load a default format. This function should validate all
> > > the fields one by one and adjust them to something appropriate.
> > >
> >
> > For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> > a. width
> > b. height
> > c. pixelformat
> > d. plane_fmt
> >     - sizeimage
> >     - bytesperline
> > e. num_planes
> > Other fields are consider constant.
> >
> > So if the user space passes one pixel format with un-supported, we will
> > apply the default format firstly and adjust width, height, sizeimage,
> > and bytesperline. We will focus on validate width & height.
> > Is it ok?
> 
> I'm not sure I understand your proposal, but let me describe the
> proper behavior here:
> 
> if (pixelformat is invalid)
>     pixelformat = some valid pixel format;
> 
> width = clamp(width, driver min, driver max);
> height = clamp(height, driver min, driver max);
> 
> num_planes = 1;
> 
> calculate_sizeimage_and_bytesperline(fmt);
> 
> fill_in_the_remaining_constant_fields(fmt);
> 
> Does it make sense?
> 
> [snip]

Yes, here is our new version.

static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
				  struct v4l2_format *f)
{
	struct mtk_cam_dev *cam = video_drvdata(file);
	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
	struct device *dev = cam->dev;
	const struct v4l2_format *dev_fmt;
	struct v4l2_format try_fmt;

	dev_dbg(dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
		__func__,
		(f->fmt.pix_mp.pixelformat & 0xFF),
		(f->fmt.pix_mp.pixelformat >> 8) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 16) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 24) & 0xFF,
		f->fmt.pix_mp.width, f->fmt.pix_mp.height);

	memset(&try_fmt, 0, sizeof(try_fmt));
	try_fmt.type = f->type;

	/* Validate pixelformat */
	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
	if (!dev_fmt) {
		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
	}
	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;

	/* Validate image width & height range */
	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);

	/* 4 bytes alignment for width */
	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);

	/* bytesperline & sizeimage calculation */
	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);

	/* Constant format fields */
	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
	try_fmt.fmt.pix_mp.num_planes = 1;
	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;

	*f = try_fmt;

	return 0;
}

> > > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > > +                           struct v4l2_format *f)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (cam_dev->streaming)
> > > > +           return -EBUSY;
> > >
> > > I think this should rather be something like vb2_queue_is_busy(), which
> > > would prevent format changes if buffers are allocated.
> > >
> >
> > Since vb2_queue_is_busy is static function, would we paste its
> > implementation in this function to check like this?
> >
> >         if (node->vdev.queue->owner &&
> >                 node->vdev.queue->owner != file->private_data) {
> >                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
> >                 return -EBUSY;
> >         }
> >
> 
> Sorry, I mixed up the function name. That should've been vb2_is_busy().
> 
> [snip]

Got it. Thanks for your suggestion.

> > > > +   /* Total pad numbers is video devices + one seninf pad */
> > > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > > +   unsigned int i;
> > > > +   int ret;
> > > > +
> > > > +   ret = mtk_cam_media_register(dev,
> > > > +                                &cam_dev->media_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > > +           return ret;
> > > > +   }
> > > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> > >
> > > An info message should be useful to the user in some way. Printing kernel
> > > pointers isn't useful. Something like "registered media0" could be useful to
> > > let the user know which media device is associated with this driver if there
> > > is more than one in the system.
> > >
> >
> > Here is the new log info.
> >
> > dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
> >
> 
> Let's fix the missing space and making a bit more readable:
> 
> dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);
> 

Ok, we will apply this change.

> >
> > > > +
> > > > +   /* Set up v4l2 device */
> > > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > > +           goto fail_v4l2_dev;
> > >
> > > Please call the labels after the cleanup step that needs to be done. It
> > > makes it easier to spot any ordering errors.
> > >
> >
> > Will fix in next patch.
> >
> > > > +   }
> > > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> > >
> > > Same as above.
> > >
> >
> > Ditto.
> >
> > dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
> >
> 
> Perhaps just "Registered %s" to be consistent with the above media log?
> 
> [snip]

Ditto.

> > > > +
> > > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > > +                                 struct v4l2_subdev *sd,
> > > > +                                 struct v4l2_async_subdev *asd)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > > +
> > >
> > > Should we somehow check that the entity we got is seninf indeed and there
> > > was no mistake in DT?
> > >
> >
> > How about to check the entity function of seninf device?
> >
> > if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >                 return -ENODEV;
> > }
> >
> > If we need to check DT, may we need to implement this in parse_endpoint
> > callback function of v4l2_async_notifier_parse_fwnode_endpoints?
> >
> 
> Yes, checking the entity function is indeed the right way to do this.
> 
> [snip]

Thanks for your confirm.

> > > > +           .default_fmt_idx = 4,
> > > > +           .max_buf_count = 10,
> > >
> > > Where does this number come from?
> > >
> >
> > The default maximum VB2 buffer count is 32.
> > In order to limit memory usage, we like to limit the maximum buffer
> > counts in the driver layer. The maximum buffer count is estimated
> > according to our camera MW.
> >
> > #define VB2_MAX_FRAME   (32)
> >
> 
> Okay, thanks.
> 
> [snip]
> > > > +   struct media_pad vdev_pad;
> > > > +   struct vb2_queue vbq;
> > > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > > +   struct list_head pending_list;
> > > > +   /* Used for vbq & vdev */
> > >
> > > It's already documented in the kerneldoc comment.
> > >
> >
> > Fixed in next patch.
> > Btw, if we remove this, we will got complain from checkpatch.pl script.
> >
> 
> Oh really, that's weird. Okay, please keep it then, sorry for the confusion.
> 
> Best regards,
> Tomasz

Ok, thanks for your understanding.
We will rollback this change to avoid checkpatch's complains.


Best regards,

Jungo



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-24  4:31             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-24  4:31 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > +                                 struct media_request *new_req)
> > > > +{
> > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +
> > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > +
> > > > +   if (!cam_dev->streaming) {
> > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > +           spin_lock(&cam_dev->req_lock);
> > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > +           spin_unlock(&cam_dev->req_lock);
> > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Normal enqueue flow */
> > > > +   if (new_req) {
> > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   /* Flush all media requests wehen first stream on */
> > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > +           list_del(&req->list);
> > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > +   }
> > > > +}
> > >
> > > This will have to be redone, as per the other suggestions, but generally one
> > > would have a function that tries to queue as much as possible from a list to
> > > the hardware and another function that adds a request to the list and calls
> > > the first function.
> > >
> >
> > We revised this function as below.
> > First to check the en-queue conditions:
> > a. stream on
> > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > at the same time.
> >
> >
> > Second, try to en-queue the frames in the pending job if possible and
> > move them into running job list if possible.
> >
> > The request has been inserted into pending job in mtk_cam_req_validate
> > which is used to validate media_request.
> 
> Thanks for replying to each of the comments, that's very helpful.
> Snipped out the parts that I agreed with.
> 
> Please note that req_validate is not supposed to change any driver
> state. It's only supposed to validate the request. req_queue is the
> right callback to insert the request into some internal driver
> bookkeeping structures.
> 

Yes, in req_validate function, we don't change any driver state.
Below is the function's implementation.

a. Call vb2_request_validate(req) to verify media request.
b. Update the buffer internal structure buffer.
c. Insert the request into pending_job_list to prepare en-queue.

static int mtk_cam_req_validate(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);
	struct media_request_object *req_obj;
	unsigned long flags;
	int ret;

	/* run buffer prepare function to initialize buffer DMA address */
	ret = vb2_request_validate(req);
	if (ret) {
		dev_err(cam->dev, "vb2_request_validate failed:%d\n", ret);
		return ret;
	}

	/* update frame_params */
	list_for_each_entry(req_obj, &req->objects, list) {
		struct vb2_buffer *vb;
		struct mtk_cam_dev_buffer *buf;

		if (!vb2_request_object_is_buffer(req_obj))
			continue;

		vb = container_of(req_obj, struct vb2_buffer, req_obj);
		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
		cam_req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
		cam_req->frame_params.dma_bufs[buf->node_id].scp_addr =
			buf->scp_addr;
	}
	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	return 0;
}

> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         struct list_head enqueue_job_list;
> >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> >         unsigned long flags;
> >
> >         if (!cam_dev->streaming ||
> >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> Do we have a guarantee that cam_dev->running_job_count doesn't
> decrement between the atomic_read() above and this line?
> 

Ok, we will use cam->pending_job_lock to protect
cam_dev->running_job_count access. Below is the revised version.

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
		dev_dbg(cam->dev, "jobs are full\n");
		spin_unlock_irqrestore(&cam->pending_job_lock, flags);
		return;
	}
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		list_del(&req->list);
		spin_lock_irqsave(&cam->running_job_lock, flags);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
		spin_unlock_irqrestore(&cam->running_job_lock, flags);
		if (atomic_inc_return(&cam->running_job_count) >=
			MTK_ISP_MAX_RUNNING_JOBS)
			break;
	}
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
}

> >                 dev_dbg(cam_dev->dev, "stream off or buffers are full:%d\n",
> >                         buffer_cnt);
> >                 return;
> >         }
> >
> >         INIT_LIST_HEAD(&enqueue_job_list);
> >
> >         spin_lock(&cam_dev->pending_job_lock);
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &cam_dev->pending_job_list, list) {
> >                 list_del(&req->list);
> >                 list_add_tail(&req->list, &enqueue_job_list);
> 
> What's the reason to use the second list? Could we just take one job
> from pending_job_list, enqueue it and then iterate again?
> 

Yes, we could simply the code block to remove enqueue_job_list.

> >                 if (atomic_inc_return(&cam_dev->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> >         }
> >         spin_unlock(&cam_dev->pending_job_lock);
> >
> >         list_for_each_entry_safe(req, req_prev,
> >                                  &enqueue_job_list, list) {
> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam_dev->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam_dev->running_job_list);
> >                 spin_unlock_irqrestore(&cam_dev->running_job_lock, flags);
> >
> 
> Do we have a guarantee that another thread doesn't run the same
> function ending up calling mtk_isp_req_enqueue() with another request
> before this one and thus making the order of running_job_list
> incorrect?
> 

In the new implementation, we use cam->pending_job_lock to protect this
scenario.

> >                 mtk_isp_req_enqueue(cam_dev, req);
> >         }
> > }
> >
> [snip]
> > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > +
> > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > +           stride = ALIGN(stride, 4);
> > >
> > > Is it expected that only the F10 format needs this alignment?
> > >
> >
> > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > should be 4. Otherwise, it is 8. We will revise this and add more
> > comments.
> 
> That means that the B10 format also needs the extra alignment, as
> opposed to what the original code did, right?
> 

Sorry for short code snippet.
This alignment checking is only applied to F10, no B10.
If you like to check the full function, you could check this in this
link[1].

static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
node_id,
			     struct v4l2_pix_format_mplane *mp)
{
	unsigned int bpl, ppl;
	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
	unsigned int width = mp->width;

	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
		/* bayer encoding format & 2 bytes alignment */
		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
		/*
		 * The FULL-G encoding format
		 * 1 G component per pixel
		 * 1 R component per 4 pixel
		 * 1 B component per 4 pixel
		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
		 */
		ppl = DIV_ROUND_UP(width * 6, 4);
		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);

		/* 4 bytes alignment for 10 bit & others are 8 bytes */
		if (pixel_bits == 10)
			bpl = ALIGN(bpl, 4);
		else
			bpl = ALIGN(bpl, 8);
	} 

[1]
https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303

> >
> > /* 4 bytes alignment for 10 bit other are 8 bytes alignment */
> >         if (pixel_bits == 10)
> >                 bpl = ALIGN(bpl, 4);
> >         else
> >                 bpl = ALIGN(bpl, 8);
> 
> SGTM, thanks.
> 
> [snip]

Thanks for your review.

> > > > +
> > > > +static struct v4l2_subdev *
> > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > +   struct media_entity *entity;
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct v4l2_subdev *sensor;
> > >
> > > This variable would be unitialized if there is no streaming sensor. Was
> > > there no compiler warning generated for this?
> > >
> >
> > No, there is no compiler warning.
> > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > with different compiler options.
> >
> 
> Thanks. It would be useful if you could check why the compiler you're
> using doesn't show a warning here. We might be missing other
> uninitialized variables.
> 

We will feedback to your project team to check the possible reason about
compiler warning issue.

> [snip]
> > > > +}
> > > > +
> > > > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > > > +                               const struct media_pad *local,
> > > > +                               const struct media_pad *remote, u32 flags)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(entity, struct mtk_cam_dev, subdev.entity);
> > > > +   u32 pad = local->index;
> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: %d -> %d flags:0x%x\n",
> > > > +           __func__, pad, remote->index, flags);
> > > > +
> > > > +   if (pad < MTK_CAM_P1_TOTAL_NODES)
> > >
> > > I assume this check is needed, because the pads with higher indexes are not
> > > video nodes? If so, a comment would be helpful here.
> > >
> >
> > Yes, we will new comment as below.
> >
> >         /*
> >          * Check video nodes is enabled by link setup.
> >          * The pad index of video node should be less than
> >          * MTK_CAM_P1_TOTAL_NODES.
> >          */
> >         if (pad < MTK_CAM_P1_TOTAL_NODES)
> >                 cam_dev->vdev_nodes[pad].enabled =
> >                         !!(flags & MEDIA_LNK_FL_ENABLED);
> >
> 
> Could we rephrase this a bit. The comment still doesn't explain why
> the index should be less than the constant. Perhaps:
> 
> /*
>  * The video nodes exposed by the driver have pads indexes
>  * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
>  */
> 
> [snip]
> 

Thanks for your suggestion.
We will update this.

> > > > +
> > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > +           __func__,
> > > > +           node->id,
> > > > +           buf->vbb.request_fd,
> > > > +           buf->vbb.vb2_buf.index);
> > > > +
> > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > +   if (vb->vb2_queue->uses_requests)
> > > > +           return;
> > >
> > > I'd suggest removing non-request support from this driver. Even if we end up
> > > with a need to provide compatibility for non-request mode, then it should be
> > > built on top of the requests mode, so that the driver itself doesn't have to
> > > deal with two modes.
> > >
> >
> > The purpose of non-request function in this driver is needed by
> > our camera middle-ware design. It needs 3A statistics buffers before
> > image buffers en-queue. So we need to en-queue 3A statistics with
> > non-request mode in this driver. After MW got the 3A statistics data, it
> > will en-queue the images, tuning buffer and other meta buffers with
> > request mode. Based on this requirement, do you have any suggestion?
> > For upstream driver, should we only consider request mode?
> >
> 
> Where does that requirement come from? Why the timing of queuing of
> the buffers to the driver is important?
> 
> [snip]

Basically, this requirement comes from our internal camera
middle-ware/3A hal in user space. Since this is not generic requirement,
we will follow your original suggestion to keep the request mode only
and remove other non-request design in other files. For upstream driver,
it should support request mode only.

> > > > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > > > +                                  unsigned int count)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = vb2_get_drv_priv(vq);
> > > > +   struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   unsigned int node_count = cam_dev->subdev.entity.use_count;
> > > > +   int ret;
> > > > +
> > > > +   if (!node->enabled) {
> > >
> > > How is this synchronized with mtk_cam_media_link_setup()?
> > >
> >
> > We will follow your suggestion and below is our proposal for this
> > function.
> >
> > 1. Use !cam_dev->pipeline.streaming_count to decide the first node to
> > stream-on.
> > 2.a If yes, do the following steps
> >     2.a-1 Call media_pipeline_start function to prevent the link
> > configuration changes.
> >     2.a-2 Call mtk_cam_dev_init_stream function to calculate how many
> > video nodes are enabled and save it into cam_dev->enabled_node_count.
> >     2.a-3 Initialize ISP P1 HW in mtk_isp_hw_init function since end
> > user has called stream-on API
> > 2.b jump step 3.
> >
> > 3. Use cam_dev->streamed_node_count to track how many video nodes are
> > streamed by user space.
> > 4. Check all enabled video nodes are streamed or not based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 5. If yes, call s_stream on for P1 sub-device
> >
> > Do you think it is reasonable?
> >
> 
> That should work indeed.
> 
> [snip]

Ok, thanks for your confirmation.

> > > > +
> > > > +   mtk_cam_vb2_return_all_buffers(cam_dev, node, VB2_BUF_STATE_ERROR);
> > >
> > > Shouldn't we stop streaming first, so that the hardware operation is
> > > cancelled and any buffers owned by the hardware are released?
> > >
> >
> > For this function, below is the new code flow.
> >
> > 1. Check the first node to stream off based on
> > cam_dev->streamed_node_count & cam_dev->enabled_node_count.
> > 2. If yes, call all s_stream off for P1 sub-device
> > 3. Call mtk_cam_vb2_return_all_buffers for each node
> > 4. Check the last node to stream off
> > 5. If yes, call media_pipeline_stop to allow user space
> > to perform link configuration changes, such as disable link.
> >
> > But, for step 5, is it too late for end user to disable link?
> > For example, for first node, it has called stream off but
> > can't call disable link until the last node is stream off?
> >
> 
> I think that should be okay. From the userspace point of view, having
> one of the video nodes streaming implies that the related subdevice
> could be streaming as well. The links between the video nodes and the
> subdevices don't have the DYNAMIC flag, so the userspace should expect
> that it can't change any links connecting to the subdevice when the
> subdevice could be streaming.
> 

Ok, got your point. We will keep this design.

> [snip]
> > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > +{
> > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > +
> > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > +                              dev->v4l2_dev.ctrl_handler);
> > >
> > > This would end up being called multiple times, once for each video node.
> > > Instead, this should be called explicitly by the driver when it completed
> > > the request - perhaps in the frame completion handler?
> > >
> > > With that, we probably wouldn't even need this callback.
> > >
> >
> > First, if we don't implement this callback function, we will receive
> > kernel warning as below.
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> >
> > Second, this function is only be called in __vb2_queue_cancel function.
> > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > So could we just implement dummy empty function?
> >
> >  * @buf_request_complete: a buffer that was never queued to the driver
> > but is
> >  *                      associated with a queued request was canceled.
> >  *                      The driver will have to mark associated objects in the
> >  *                      request as completed; required if requests are
> >  *                      supported.
> >
> 
> Good catch, thanks.
> 
> Sounds like we may indeed need to implement this callback. In
> particular, we may need to remove the request that the buffer was
> associated with from the driver queue and return the other buffers
> associated to it with an error state. This should be similar to
> handling a request failure.
> [snip]

Before calling this callback function, the VB2's stop_streaming has been
called. Normally, we will return the buffers belonged to this vb2 queu
with error state. On other hand, only if the state of request is
MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
__vb2_queue_cancel function. It hints this media request has been
validated and inserted into our driver's pending_job_list or
running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
requests from driver's list when streaming is off. Since we have no
v4l2_ctrl, do we need to do the above things which is already handled in
mtk_cam_vb2_stop_streaming function? Maybe is this callback function
only designed for v4l2_ctrl_request_complete usage?

static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	dev_dbg(cam->dev, "%s\n", __func__);

	spin_lock_irqsave(&cam->pending_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->pending_job_lock, flags);

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
		list_del(&req->list);
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
}

static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
{
	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
	struct device *dev = cam->dev;

	dev_dbg(dev, "%s node:%d count info:%d", __func__,
		node->id, atomic_read(&cam->stream_count));

	mutex_lock(&cam->op_lock);
	if (atomic_read(&cam->stream_count) == cam->enabled_count)
		if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
			dev_err(dev, "failed to stop streaming\n");

	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);

	/* Check the first node to stream-off */
	if (!atomic_dec_and_test(&cam->stream_count)) {
		mutex_unlock(&cam->op_lock);
		return;
	}
	mutex_unlock(&cam->op_lock);

	mtk_cam_dev_req_cleanup(cam);
	media_pipeline_stop(&node->vdev.entity);
}

> > > > +
> > > > +   return 0;
> > > > +}
> > > > +
> > > > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > > > +                              struct v4l2_fmtdesc *f)
> > > > +{
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (f->index >= node->desc.num_fmts)
> > > > +           return -EINVAL;
> > > > +
> > > > +   f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > >
> > > Is the set of formats available always the same regardless of the sensor
> > > format?
> > >
> >
> > Yes, ISP P1 HW output formats are always available without impact
> > by sensor formats.
> >
> > > > +   f->flags = 0;
> > >
> > > We need f->description too.
> > >
> >
> > For this description, do you suggest 1). we fill this field in this
> > function or 2). v4l_fill_fmtdesc function in v4l2-ioctl?
> >
> > https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-ioctl.c#L1152
> >
> > Basically, we prefer method 1.
> >
> 
> That should be v4l_fill_fmtdesc(), as it already includes other
> vendor-specific formats.
> 
> [snip]


Ok, got it. We will follow your suggestion.

> > > > +
> > > > +   dev_dbg(&cam_dev->pdev->dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
> > > > +           __func__,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat & 0xFF),
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 8) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 16) & 0xFF,
> > > > +           (in_fmt->fmt.pix_mp.pixelformat >> 24) & 0xFF,
> > > > +           in_fmt->fmt.pix_mp.width, in_fmt->fmt.pix_mp.height);
> > > > +
> > > > +   width = in_fmt->fmt.pix_mp.width;
> > > > +   height = in_fmt->fmt.pix_mp.height;
> > > > +
> > > > +   dev_fmt = mtk_cam_dev_find_fmt(&node->desc,
> > > > +                                  in_fmt->fmt.pix_mp.pixelformat);
> > > > +   if (dev_fmt) {
> > > > +           mtk_cam_dev_set_img_fmt(&cam_dev->pdev->dev,
> > > > +                                   &in_fmt->fmt.pix_mp,
> > > > +                                   &dev_fmt->fmt.pix_mp,
> > > > +                                   node->id);
> > > > +   } else {
> > > > +           mtk_cam_dev_load_default_fmt(&cam_dev->pdev->dev,
> > > > +                                        &node->desc, in_fmt);
> > >
> > > We shouldn't just load a default format. This function should validate all
> > > the fields one by one and adjust them to something appropriate.
> > >
> >
> > For ISP P1 HW, we only cares these fields of v4l2_pix_format_mplane.
> > a. width
> > b. height
> > c. pixelformat
> > d. plane_fmt
> >     - sizeimage
> >     - bytesperline
> > e. num_planes
> > Other fields are consider constant.
> >
> > So if the user space passes one pixel format with un-supported, we will
> > apply the default format firstly and adjust width, height, sizeimage,
> > and bytesperline. We will focus on validate width & height.
> > Is it ok?
> 
> I'm not sure I understand your proposal, but let me describe the
> proper behavior here:
> 
> if (pixelformat is invalid)
>     pixelformat = some valid pixel format;
> 
> width = clamp(width, driver min, driver max);
> height = clamp(height, driver min, driver max);
> 
> num_planes = 1;
> 
> calculate_sizeimage_and_bytesperline(fmt);
> 
> fill_in_the_remaining_constant_fields(fmt);
> 
> Does it make sense?
> 
> [snip]

Yes, here is our new version.

static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
				  struct v4l2_format *f)
{
	struct mtk_cam_dev *cam = video_drvdata(file);
	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
	struct device *dev = cam->dev;
	const struct v4l2_format *dev_fmt;
	struct v4l2_format try_fmt;

	dev_dbg(dev, "%s: fmt:%c%c%c%c, w*h:%u*%u\n",
		__func__,
		(f->fmt.pix_mp.pixelformat & 0xFF),
		(f->fmt.pix_mp.pixelformat >> 8) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 16) & 0xFF,
		(f->fmt.pix_mp.pixelformat >> 24) & 0xFF,
		f->fmt.pix_mp.width, f->fmt.pix_mp.height);

	memset(&try_fmt, 0, sizeof(try_fmt));
	try_fmt.type = f->type;

	/* Validate pixelformat */
	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
	if (!dev_fmt) {
		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
	}
	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;

	/* Validate image width & height range */
	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);

	/* 4 bytes alignment for width */
	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);

	/* bytesperline & sizeimage calculation */
	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);

	/* Constant format fields */
	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
	try_fmt.fmt.pix_mp.num_planes = 1;
	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;

	*f = try_fmt;

	return 0;
}

> > > > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > > > +                           struct v4l2_format *f)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev = video_drvdata(file);
> > > > +   struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > > > +
> > > > +   if (cam_dev->streaming)
> > > > +           return -EBUSY;
> > >
> > > I think this should rather be something like vb2_queue_is_busy(), which
> > > would prevent format changes if buffers are allocated.
> > >
> >
> > Since vb2_queue_is_busy is static function, would we paste its
> > implementation in this function to check like this?
> >
> >         if (node->vdev.queue->owner &&
> >                 node->vdev.queue->owner != file->private_data) {
> >                 dev_err(cam_dev->dev, "%s err: buffer allocated\n", __func__);
> >                 return -EBUSY;
> >         }
> >
> 
> Sorry, I mixed up the function name. That should've been vb2_is_busy().
> 
> [snip]

Got it. Thanks for your suggestion.

> > > > +   /* Total pad numbers is video devices + one seninf pad */
> > > > +   unsigned int num_subdev_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > > > +   unsigned int i;
> > > > +   int ret;
> > > > +
> > > > +   ret = mtk_cam_media_register(dev,
> > > > +                                &cam_dev->media_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register media device:%d\n", ret);
> > > > +           return ret;
> > > > +   }
> > > > +   dev_info(dev, "Register media device: %s, 0x%pK",
> > > > +            MTK_CAM_DEV_P1_NAME, cam_dev->media_dev);
> > >
> > > An info message should be useful to the user in some way. Printing kernel
> > > pointers isn't useful. Something like "registered media0" could be useful to
> > > let the user know which media device is associated with this driver if there
> > > is more than one in the system.
> > >
> >
> > Here is the new log info.
> >
> > dev_info(dev, "media%d register",cam->media_dev.devnode->minor);
> >
> 
> Let's fix the missing space and making a bit more readable:
> 
> dev_info(dev, "Registered media%d", cam->media_dev.devnode->minor);
> 

Ok, we will apply this change.

> >
> > > > +
> > > > +   /* Set up v4l2 device */
> > > > +   cam_dev->v4l2_dev.mdev = &cam_dev->media_dev;
> > > > +   ret = v4l2_device_register(dev, &cam_dev->v4l2_dev);
> > > > +   if (ret) {
> > > > +           dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > > > +           goto fail_v4l2_dev;
> > >
> > > Please call the labels after the cleanup step that needs to be done. It
> > > makes it easier to spot any ordering errors.
> > >
> >
> > Will fix in next patch.
> >
> > > > +   }
> > > > +   dev_info(dev, "Register v4l2 device: 0x%pK", cam_dev->v4l2_dev);
> > >
> > > Same as above.
> > >
> >
> > Ditto.
> >
> > dev_info(dev, "Register v4l2 device: %s", cam->v4l2_dev.name);
> >
> 
> Perhaps just "Registered %s" to be consistent with the above media log?
> 
> [snip]

Ditto.

> > > > +
> > > > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > > > +                                 struct v4l2_subdev *sd,
> > > > +                                 struct v4l2_async_subdev *asd)
> > > > +{
> > > > +   struct mtk_cam_dev *cam_dev =
> > > > +           container_of(notifier, struct mtk_cam_dev, notifier);
> > > > +
> > >
> > > Should we somehow check that the entity we got is seninf indeed and there
> > > was no mistake in DT?
> > >
> >
> > How about to check the entity function of seninf device?
> >
> > if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >         dev_dbg(cam->dev, "No MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >                 return -ENODEV;
> > }
> >
> > If we need to check DT, may we need to implement this in parse_endpoint
> > callback function of v4l2_async_notifier_parse_fwnode_endpoints?
> >
> 
> Yes, checking the entity function is indeed the right way to do this.
> 
> [snip]

Thanks for your confirm.

> > > > +           .default_fmt_idx = 4,
> > > > +           .max_buf_count = 10,
> > >
> > > Where does this number come from?
> > >
> >
> > The default maximum VB2 buffer count is 32.
> > In order to limit memory usage, we like to limit the maximum buffer
> > counts in the driver layer. The maximum buffer count is estimated
> > according to our camera MW.
> >
> > #define VB2_MAX_FRAME   (32)
> >
> 
> Okay, thanks.
> 
> [snip]
> > > > +   struct media_pad vdev_pad;
> > > > +   struct vb2_queue vbq;
> > > > +   struct v4l2_ctrl_handler ctrl_handler;
> > > > +   struct list_head pending_list;
> > > > +   /* Used for vbq & vdev */
> > >
> > > It's already documented in the kerneldoc comment.
> > >
> >
> > Fixed in next patch.
> > Btw, if we remove this, we will got complain from checkpatch.pl script.
> >
> 
> Oh really, that's weird. Okay, please keep it then, sorry for the confusion.
> 
> Best regards,
> Tomasz

Ok, thanks for your understanding.
We will rollback this change to avoid checkpatch's complains.


Best regards,

Jungo



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-07-20  9:58         ` Jungo Lin
  (?)
@ 2019-07-25  9:23           ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25  9:23 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab

.Hi Jungo,

On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > It handles the ISP HW configuration, provides interrupt handling and
> > > initializes the V4L2 device nodes and other functions.
> > >
> > > (The current metadata interface used in meta input and partial
> > > meta nodes is only a temporary solution to kick off the driver
> > > development and is not ready to be reviewed yet.)
> > >
> > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > ---
> > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > >  4 files changed, 1457 insertions(+)
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > >
> >
> > Thanks for the patch! Please see my comments inline.
> >
> > [snip]
> >
>
> Thanks for your comments. Please check my replies inline.
>

Thanks! I'll snip anything I don't have further comments on.

[snip]
> > > +/* META */
> > > +#define REG_META0_VB2_INDEX                0x14dc
> > > +#define REG_META1_VB2_INDEX                0x151c
> >
> > I don't believe these registers are really for VB2 indexes.
> >
>
> MTK P1 ISP HW supports frame header spare registers for each DMA, such
> as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> frame information in these ISP registers. In this case, we save META0
> VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> spare register. These implementation is designed for non-request 3A
> DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> REG_META0_VB2_INDEX for easy understanding.

Unfortunately it's not a good idea to mix hardware concepts with
naming specific to the OS the driver is written for. Better to keep
the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.

> Moreover, if we only need to
> support request mode, we should remove this here.
>
> cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> cmd_params.meta_frame.enabled_dma = dma_port;
> cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
>

Okay, removing sounds good to me. Let's keep the code simple.

[snip]
> > > +
> > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > +
> > > +   /* Sof, done order check */
> > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > +
> > > +           /* Notify IRQ event and enqueue frame */
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > +           isp_dev->current_frame = hw_frame_num;
> >
> > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > event?
> >
>
> This is a another spare register for frame sequence number usage.
> It comes from struct p1_frame_param:frame_seq_no which is sent by
> SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> Is it a better understanding?

I'm sorry, unfortunately it's still not clear to me. Is it the
sequence number of the frame that was just processed and returned to
the kernel or the next frame that is going to be processed from now
on?

>
> Below is our frame request handling in current design.
>
> 1. Buffer preparation
> - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> other meta histogram buffers (LCSO/LMVO) into one request.
> - Accumulated one unique frame sequence number to each request and send
> this request to the SCP composer to compose CQ (Command queue) buffer
> via SCP_ISP_FRAME IPI command.
> - CQ buffer is frame registers set. If ISP registers should be updated
> per frame, these registers are configured in the CQ buffer, such as
> frame sequence number, DMA addresses and tuning ISP registers.
> - One frame request will be composed into one CQ buffer.Once CQ buffer
> is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> its corresponding frame sequence number. Based on this, kernel driver
> knows which request is ready to be en-queued and save this with
> p1_dev->isp_ctx.composed_frame_id.

Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
already have a linked lists of requests that are composed and ready to
be enqueued? Also, the request itself would contain its frame ID
inside the driver request struct, right?

> - The maximum number of CQ buffers in SCP is 3.
>
> 2. Buffer en-queue flow
> - In order to configure correct CQ buffer setting before next SQF event,
> it is depended on by MTK ISP P1 HW CQ mechanism.
> - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> when HW_PASS1_DON_ST is received which means DMA output is done.
> - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> address is used. Otherwise, it will loaded one dummy CQ buffer to
> bypass.
> - So we will check available CQ buffers by comparing composed frame
> sequence number & dequeued frame sequence from ISP HW in SOF event.
> - If there are available CQ buffers, update the CQ base address to the
> next CQ buffer address based on current de-enqueue frame sequence
> number. So MTK ISP P1 HW will load this CQ buffer into HW when
> HW_PASS1_DON_ST is triggered which is before the next SOF.
> - So in next SOF event, ISP HW starts to output DMA buffers with this
> request until request is done.
> - But, for the first request, it is loaded into HW manually when
> streaming is on for better performance.
>
> 3. Buffer de-queue flow
> - We will use frame sequence number to decide which request is ready to
> de-queue.
> - We will save some important register setting from ISP HW when SOF is
> received. This is because the ISP HW starts to output the data with the
> corresponding settings, especially frame sequence number setting.

Could you explain a bit more about these important register settings?
When does the hardware update the values in the register to new ones?
At SOF?

> - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> done. So we could call isp_deque_request_frame with frame sequence
> number to de-queue frame to VB2

What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?

> - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> AAO/AFO non-request buffers are en-queued without request buffers at the
> same time, there will be no SW P1 done event for AAO/AFO DMA done.
> Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.

Do we have a case like this? Wouldn't we normally always want to
bundle AAO/AFO buffers with frame buffers?

> - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> we may try to send another request to SCP for composing.

Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
buffers, right?

>
> Hopefully, my explanation is helpful for better understanding our
> implementation. If you still have any questions, please let me know.
>

Yes, it's more clear now, thanks. Still some more comments above, though.

> > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +   } else {
> > > +           if (irq_status & SOF_INT_ST) {
> > > +                   isp_dev->current_frame = hw_frame_num;
> > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +           }
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > +   }
> >
> > The if and else blocks do almost the same things just in different order. Is
> > it really expected?
> >
>
> If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> any de-queued frame and update the next frame setting later.
> Normally, this is a corner case or system performance issue.

So it sounds like HW_PASS1_DON_ST means that all data from current
frame has been written, right? If I understand your explanation above
correctly, that would mean following handling of each interrupt:

HW_PASS1_DON_ST:
 - CQ executes with next CQ buffer to prepare for next frame. <- how
is this handled? does the CQ hardware automatically receive this event
from the ISP hadware?
 - return VB2 buffers,
 - complete requests.

SOF_INT_ST:
 - send VSYNC event to userspace,
 - program next CQ buffer to CQ,

SW_PASS1_DON_ST:
 - reclaim CQ buffer and enqueue next frame to composing if available

>
> Btw, we will revise the above implementation as below.
>
>
> if (irq_status & SOF_INT_ST)
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
>                                              dequeue_frame_seq_no);
>
> /* Sof, done order check */
> if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
> /* Notify IRQ event and de-enqueue frame */
> irq_handle_notify_event(p1_dev, irq_status, dma_status);

Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?

[snip]
> > > +/* ISP P1 interface functions */
> > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   int ret;
> > > +
> > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> > > +
> > > +   ret = isp_init_context(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> >
> > The above function doesn't really seem to be related to power management.
> > Should it be called from subdev stream on?
> >
>
> We will rename this function to mtk_isp_hw_init.
> But, it will be called when the first video node is streamed on.
> This is because we need to initialize the HW firstly for sub-device
> stream-on performance.  We need to send some IPI commands, such as
> ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> timing.

What performance do you mean here? The time between first video node
stream on and last video node stream on should be really short. Are
you seeing some long delays there?

That said, doing it when the first video node starts streaming is okay.

[snip]
> > > +   /* Use pure RAW as default HW path */
> > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > +
> > > +   isp_composer_hw_init(dev);
> > > +   /* Check enabled DMAs which is configured by media setup */
> > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> >
> > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > necessary?
> >
>
> Yes, it is necessary for non-request buffers design for Camera 3A video
> nodes. For 3A video nodes, we just want to know which 3A video nodes are
> enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> format from user space. So we just pass the enabled DMA information from
> kernel to SCP only. With 3A enabled DMA information, we could configure
> related 3A registers in SCP.

We should try to remove this non-request mode. Let's continue
discussion on the other patch where I brought this topic.

[snip]
> > > +int mtk_isp_power_release(struct device *dev)
> > > +{
> > > +   isp_composer_hw_deinit(dev);
> > > +   isp_uninit_context(dev);
> >
> > These two don't seem to be related to power either.
> >
> > Instead, I don't see anything that could undo the rproc_boot() operation
> > here.
> >
>
> We will rename this function to mtk_isp_hw_release.
> We will also add rproc_shutdown function call here.
>
> int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> {
>         struct device *dev = cam->dev;
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         isp_composer_hw_deinit(p1_dev);
>         pm_runtime_put_sync_autosuspend(dev);

Note that for autosuspend to work correctly, you also need to call
pm_runtime_mark_last_busy() before this one.

[snip]
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   struct p1_frame_param frameparams;
> > > +   struct mtk_isp_queue_job *framejob;
> > > +   struct media_request_object *obj, *obj_safe;
> > > +   struct vb2_buffer *vb;
> > > +   struct mtk_cam_dev_buffer *buf;
> > > +
> > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> >
> > This allocation shouldn't be needed. The structure should be already a part
> > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > even say that the contents of this struct should be just moved to that one
> > to avoid overabstracting.
> >
>
> For this function, we will apply the new design from P2 driver's
> comment. Here is the new implementation.
>
> struct mtk_cam_dev_request {
>         struct media_request req;
>         struct mtk_p1_frame_param frame_params;
>         struct work_struct frame_work;
>         struct list_head list;
>         atomic_t buf_count;
> };
>
> void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>                          struct mtk_cam_dev_request *req)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>         int ret;
>
>         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
>         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
>         if (!ret)
>                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
>                         req->frame_params.frame_seq_no, ret);
>         else
>                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>                         req->req.debug_str, req->frame_params.frame_seq_no,
>                         atomic_read(&cam->running_job_count));

It shouldn't be possible for queue_work() to fail here. We just
received a brand new request from the Request API core and called
INIT_WORK() on the work struct.

[snip]
> > > +   enable_sys_clock(p1_dev);
> > > +
> > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > +   if (p1_dev->cam_dev.streaming) {
> > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > +           /* Enable CMOS */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > +           writel((reg_val | CMOS_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > +           /* Enable VF */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > +           writel((reg_val | VFDATA_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > +   }
> >
> > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > Would the code above continue all the capture from the next buffer, as
> > queued by the userspace before the suspend?
> >
>
> Yes, we will test it.
> 1. SCP buffers are kept by SCP processor
> 2. ISP registers are still kept even if ISP clock is disable.
>

That said, during system suspend, it would be more than just ISP clock
disabled. I'd expect that the ISP power domain would be powered off.
However, if we ensure that the ISP completes before suspend, I guess
that after the resume the next frame CQ buffer would reprogram all the
registers, right?

Also, would SCP always keep running in system suspend?

[snip]
> > > +
> > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> >
> > I think we want to start from 0 here?
> >
>
> Because of single CAM support, we will revise our DTS tree to support
> single CAM only.

Note that DT bindings should describe the hardware not the driver. So
please design the bindings in a way that would cover all the cameras,
even if the driver only takes the information needed to handle 1.

> So we could remove this loop and check the CAM-B HW
> information here. Here is below new function.
>
> static int mtk_isp_probe(struct platform_device *pdev)
> {
>         /* List of clocks required by isp cam */
>         static const char * const clk_names[] = {
>                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
>         };
>         struct mtk_isp_p1_device *p1_dev;
>         struct device *dev = &pdev->dev;
>         struct resource *res;
>         int irq, ret, i;
>
>         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
>         if (!p1_dev)
>                 return -ENOMEM;
>
>         p1_dev->dev = dev;
>         dev_set_drvdata(dev, p1_dev);
>
>         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>         p1_dev->regs = devm_ioremap_resource(dev, res);
>         if (IS_ERR(p1_dev->regs)) {
>                 dev_err(dev, "Failed platform resources map\n");
>                 return PTR_ERR(p1_dev->regs);
>         }
>         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
>
>         irq = platform_get_irq(pdev, 0);
>         if (!irq) {
>                 dev_err(dev, "Missing IRQ resources data\n");
>                 return -ENODEV;
>         }
>         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
>                                dev_name(dev), p1_dev);
>         if (ret) {
>                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
>                         dev->of_node->name, irq);
>                 return ret;
>         }
>         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
>         spin_lock_init(&p1_dev->spinlock_irq);
>
>         p1_dev->num_clks = ARRAY_SIZE(clk_names);
>         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
>                                     sizeof(*p1_dev->clks), GFP_KERNEL);
>         if (!p1_dev->clks)
>                 return -ENOMEM;
>
>         for (i = 0; i < p1_dev->num_clks; ++i)
>                 p1_dev->clks[i].id = clk_names[i];
>
>         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
>                 return ret;
>         }
>
>         ret = isp_setup_scp_rproc(p1_dev, pdev);
>         if (ret)
>                 return ret;
>
>         pm_runtime_enable(dev);

We also need to call pm_runtime_use_autosuspend() and
pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
suggest an autosuspend delay equal to around 2x the time that's needed
to stop and start streaming in total.

[snip]
> > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> >
> > For V4L2 drivers system and runtime PM ops would normally be completely
> > different. Runtime PM ops would be called when the hardware is idle already
> > or is about to become active. System PM ops would be called at system power
> > state change and the hardware might be both idle or active. Please also see
> > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> >
>
> Here is the new implementation. It should be clear to show the
> difference between system and runtime PM ops.
>
> static const struct dev_pm_ops mtk_isp_pm_ops = {
>         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
>                                 pm_runtime_force_resume)
>         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> NULL)
> };

That's still not correct. In runtime suspend/resume ops we already are
not streaming anymore, because we call pm_runtime_get/put_*() when
starting and stopping streaming. In system suspend/resume ops we might
be streaming and that's when we need to stop the hardware and wait for
it to finish. Please implement these ops separately.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-25  9:23           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25  9:23 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

.Hi Jungo,

On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > It handles the ISP HW configuration, provides interrupt handling and
> > > initializes the V4L2 device nodes and other functions.
> > >
> > > (The current metadata interface used in meta input and partial
> > > meta nodes is only a temporary solution to kick off the driver
> > > development and is not ready to be reviewed yet.)
> > >
> > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > ---
> > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > >  4 files changed, 1457 insertions(+)
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > >
> >
> > Thanks for the patch! Please see my comments inline.
> >
> > [snip]
> >
>
> Thanks for your comments. Please check my replies inline.
>

Thanks! I'll snip anything I don't have further comments on.

[snip]
> > > +/* META */
> > > +#define REG_META0_VB2_INDEX                0x14dc
> > > +#define REG_META1_VB2_INDEX                0x151c
> >
> > I don't believe these registers are really for VB2 indexes.
> >
>
> MTK P1 ISP HW supports frame header spare registers for each DMA, such
> as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> frame information in these ISP registers. In this case, we save META0
> VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> spare register. These implementation is designed for non-request 3A
> DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> REG_META0_VB2_INDEX for easy understanding.

Unfortunately it's not a good idea to mix hardware concepts with
naming specific to the OS the driver is written for. Better to keep
the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.

> Moreover, if we only need to
> support request mode, we should remove this here.
>
> cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> cmd_params.meta_frame.enabled_dma = dma_port;
> cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
>

Okay, removing sounds good to me. Let's keep the code simple.

[snip]
> > > +
> > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > +
> > > +   /* Sof, done order check */
> > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > +
> > > +           /* Notify IRQ event and enqueue frame */
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > +           isp_dev->current_frame = hw_frame_num;
> >
> > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > event?
> >
>
> This is a another spare register for frame sequence number usage.
> It comes from struct p1_frame_param:frame_seq_no which is sent by
> SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> Is it a better understanding?

I'm sorry, unfortunately it's still not clear to me. Is it the
sequence number of the frame that was just processed and returned to
the kernel or the next frame that is going to be processed from now
on?

>
> Below is our frame request handling in current design.
>
> 1. Buffer preparation
> - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> other meta histogram buffers (LCSO/LMVO) into one request.
> - Accumulated one unique frame sequence number to each request and send
> this request to the SCP composer to compose CQ (Command queue) buffer
> via SCP_ISP_FRAME IPI command.
> - CQ buffer is frame registers set. If ISP registers should be updated
> per frame, these registers are configured in the CQ buffer, such as
> frame sequence number, DMA addresses and tuning ISP registers.
> - One frame request will be composed into one CQ buffer.Once CQ buffer
> is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> its corresponding frame sequence number. Based on this, kernel driver
> knows which request is ready to be en-queued and save this with
> p1_dev->isp_ctx.composed_frame_id.

Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
already have a linked lists of requests that are composed and ready to
be enqueued? Also, the request itself would contain its frame ID
inside the driver request struct, right?

> - The maximum number of CQ buffers in SCP is 3.
>
> 2. Buffer en-queue flow
> - In order to configure correct CQ buffer setting before next SQF event,
> it is depended on by MTK ISP P1 HW CQ mechanism.
> - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> when HW_PASS1_DON_ST is received which means DMA output is done.
> - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> address is used. Otherwise, it will loaded one dummy CQ buffer to
> bypass.
> - So we will check available CQ buffers by comparing composed frame
> sequence number & dequeued frame sequence from ISP HW in SOF event.
> - If there are available CQ buffers, update the CQ base address to the
> next CQ buffer address based on current de-enqueue frame sequence
> number. So MTK ISP P1 HW will load this CQ buffer into HW when
> HW_PASS1_DON_ST is triggered which is before the next SOF.
> - So in next SOF event, ISP HW starts to output DMA buffers with this
> request until request is done.
> - But, for the first request, it is loaded into HW manually when
> streaming is on for better performance.
>
> 3. Buffer de-queue flow
> - We will use frame sequence number to decide which request is ready to
> de-queue.
> - We will save some important register setting from ISP HW when SOF is
> received. This is because the ISP HW starts to output the data with the
> corresponding settings, especially frame sequence number setting.

Could you explain a bit more about these important register settings?
When does the hardware update the values in the register to new ones?
At SOF?

> - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> done. So we could call isp_deque_request_frame with frame sequence
> number to de-queue frame to VB2

What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?

> - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> AAO/AFO non-request buffers are en-queued without request buffers at the
> same time, there will be no SW P1 done event for AAO/AFO DMA done.
> Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.

Do we have a case like this? Wouldn't we normally always want to
bundle AAO/AFO buffers with frame buffers?

> - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> we may try to send another request to SCP for composing.

Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
buffers, right?

>
> Hopefully, my explanation is helpful for better understanding our
> implementation. If you still have any questions, please let me know.
>

Yes, it's more clear now, thanks. Still some more comments above, though.

> > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +   } else {
> > > +           if (irq_status & SOF_INT_ST) {
> > > +                   isp_dev->current_frame = hw_frame_num;
> > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +           }
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > +   }
> >
> > The if and else blocks do almost the same things just in different order. Is
> > it really expected?
> >
>
> If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> any de-queued frame and update the next frame setting later.
> Normally, this is a corner case or system performance issue.

So it sounds like HW_PASS1_DON_ST means that all data from current
frame has been written, right? If I understand your explanation above
correctly, that would mean following handling of each interrupt:

HW_PASS1_DON_ST:
 - CQ executes with next CQ buffer to prepare for next frame. <- how
is this handled? does the CQ hardware automatically receive this event
from the ISP hadware?
 - return VB2 buffers,
 - complete requests.

SOF_INT_ST:
 - send VSYNC event to userspace,
 - program next CQ buffer to CQ,

SW_PASS1_DON_ST:
 - reclaim CQ buffer and enqueue next frame to composing if available

>
> Btw, we will revise the above implementation as below.
>
>
> if (irq_status & SOF_INT_ST)
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
>                                              dequeue_frame_seq_no);
>
> /* Sof, done order check */
> if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
> /* Notify IRQ event and de-enqueue frame */
> irq_handle_notify_event(p1_dev, irq_status, dma_status);

Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?

[snip]
> > > +/* ISP P1 interface functions */
> > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   int ret;
> > > +
> > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> > > +
> > > +   ret = isp_init_context(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> >
> > The above function doesn't really seem to be related to power management.
> > Should it be called from subdev stream on?
> >
>
> We will rename this function to mtk_isp_hw_init.
> But, it will be called when the first video node is streamed on.
> This is because we need to initialize the HW firstly for sub-device
> stream-on performance.  We need to send some IPI commands, such as
> ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> timing.

What performance do you mean here? The time between first video node
stream on and last video node stream on should be really short. Are
you seeing some long delays there?

That said, doing it when the first video node starts streaming is okay.

[snip]
> > > +   /* Use pure RAW as default HW path */
> > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > +
> > > +   isp_composer_hw_init(dev);
> > > +   /* Check enabled DMAs which is configured by media setup */
> > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> >
> > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > necessary?
> >
>
> Yes, it is necessary for non-request buffers design for Camera 3A video
> nodes. For 3A video nodes, we just want to know which 3A video nodes are
> enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> format from user space. So we just pass the enabled DMA information from
> kernel to SCP only. With 3A enabled DMA information, we could configure
> related 3A registers in SCP.

We should try to remove this non-request mode. Let's continue
discussion on the other patch where I brought this topic.

[snip]
> > > +int mtk_isp_power_release(struct device *dev)
> > > +{
> > > +   isp_composer_hw_deinit(dev);
> > > +   isp_uninit_context(dev);
> >
> > These two don't seem to be related to power either.
> >
> > Instead, I don't see anything that could undo the rproc_boot() operation
> > here.
> >
>
> We will rename this function to mtk_isp_hw_release.
> We will also add rproc_shutdown function call here.
>
> int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> {
>         struct device *dev = cam->dev;
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         isp_composer_hw_deinit(p1_dev);
>         pm_runtime_put_sync_autosuspend(dev);

Note that for autosuspend to work correctly, you also need to call
pm_runtime_mark_last_busy() before this one.

[snip]
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   struct p1_frame_param frameparams;
> > > +   struct mtk_isp_queue_job *framejob;
> > > +   struct media_request_object *obj, *obj_safe;
> > > +   struct vb2_buffer *vb;
> > > +   struct mtk_cam_dev_buffer *buf;
> > > +
> > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> >
> > This allocation shouldn't be needed. The structure should be already a part
> > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > even say that the contents of this struct should be just moved to that one
> > to avoid overabstracting.
> >
>
> For this function, we will apply the new design from P2 driver's
> comment. Here is the new implementation.
>
> struct mtk_cam_dev_request {
>         struct media_request req;
>         struct mtk_p1_frame_param frame_params;
>         struct work_struct frame_work;
>         struct list_head list;
>         atomic_t buf_count;
> };
>
> void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>                          struct mtk_cam_dev_request *req)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>         int ret;
>
>         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
>         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
>         if (!ret)
>                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
>                         req->frame_params.frame_seq_no, ret);
>         else
>                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>                         req->req.debug_str, req->frame_params.frame_seq_no,
>                         atomic_read(&cam->running_job_count));

It shouldn't be possible for queue_work() to fail here. We just
received a brand new request from the Request API core and called
INIT_WORK() on the work struct.

[snip]
> > > +   enable_sys_clock(p1_dev);
> > > +
> > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > +   if (p1_dev->cam_dev.streaming) {
> > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > +           /* Enable CMOS */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > +           writel((reg_val | CMOS_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > +           /* Enable VF */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > +           writel((reg_val | VFDATA_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > +   }
> >
> > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > Would the code above continue all the capture from the next buffer, as
> > queued by the userspace before the suspend?
> >
>
> Yes, we will test it.
> 1. SCP buffers are kept by SCP processor
> 2. ISP registers are still kept even if ISP clock is disable.
>

That said, during system suspend, it would be more than just ISP clock
disabled. I'd expect that the ISP power domain would be powered off.
However, if we ensure that the ISP completes before suspend, I guess
that after the resume the next frame CQ buffer would reprogram all the
registers, right?

Also, would SCP always keep running in system suspend?

[snip]
> > > +
> > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> >
> > I think we want to start from 0 here?
> >
>
> Because of single CAM support, we will revise our DTS tree to support
> single CAM only.

Note that DT bindings should describe the hardware not the driver. So
please design the bindings in a way that would cover all the cameras,
even if the driver only takes the information needed to handle 1.

> So we could remove this loop and check the CAM-B HW
> information here. Here is below new function.
>
> static int mtk_isp_probe(struct platform_device *pdev)
> {
>         /* List of clocks required by isp cam */
>         static const char * const clk_names[] = {
>                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
>         };
>         struct mtk_isp_p1_device *p1_dev;
>         struct device *dev = &pdev->dev;
>         struct resource *res;
>         int irq, ret, i;
>
>         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
>         if (!p1_dev)
>                 return -ENOMEM;
>
>         p1_dev->dev = dev;
>         dev_set_drvdata(dev, p1_dev);
>
>         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>         p1_dev->regs = devm_ioremap_resource(dev, res);
>         if (IS_ERR(p1_dev->regs)) {
>                 dev_err(dev, "Failed platform resources map\n");
>                 return PTR_ERR(p1_dev->regs);
>         }
>         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
>
>         irq = platform_get_irq(pdev, 0);
>         if (!irq) {
>                 dev_err(dev, "Missing IRQ resources data\n");
>                 return -ENODEV;
>         }
>         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
>                                dev_name(dev), p1_dev);
>         if (ret) {
>                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
>                         dev->of_node->name, irq);
>                 return ret;
>         }
>         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
>         spin_lock_init(&p1_dev->spinlock_irq);
>
>         p1_dev->num_clks = ARRAY_SIZE(clk_names);
>         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
>                                     sizeof(*p1_dev->clks), GFP_KERNEL);
>         if (!p1_dev->clks)
>                 return -ENOMEM;
>
>         for (i = 0; i < p1_dev->num_clks; ++i)
>                 p1_dev->clks[i].id = clk_names[i];
>
>         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
>                 return ret;
>         }
>
>         ret = isp_setup_scp_rproc(p1_dev, pdev);
>         if (ret)
>                 return ret;
>
>         pm_runtime_enable(dev);

We also need to call pm_runtime_use_autosuspend() and
pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
suggest an autosuspend delay equal to around 2x the time that's needed
to stop and start streaming in total.

[snip]
> > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> >
> > For V4L2 drivers system and runtime PM ops would normally be completely
> > different. Runtime PM ops would be called when the hardware is idle already
> > or is about to become active. System PM ops would be called at system power
> > state change and the hardware might be both idle or active. Please also see
> > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> >
>
> Here is the new implementation. It should be clear to show the
> difference between system and runtime PM ops.
>
> static const struct dev_pm_ops mtk_isp_pm_ops = {
>         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
>                                 pm_runtime_force_resume)
>         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> NULL)
> };

That's still not correct. In runtime suspend/resume ops we already are
not streaming anymore, because we call pm_runtime_get/put_*() when
starting and stopping streaming. In system suspend/resume ops we might
be streaming and that's when we need to stop the hardware and wait for
it to finish. Please implement these ops separately.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-25  9:23           ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25  9:23 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

.Hi Jungo,

On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > It handles the ISP HW configuration, provides interrupt handling and
> > > initializes the V4L2 device nodes and other functions.
> > >
> > > (The current metadata interface used in meta input and partial
> > > meta nodes is only a temporary solution to kick off the driver
> > > development and is not ready to be reviewed yet.)
> > >
> > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > ---
> > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > >  4 files changed, 1457 insertions(+)
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > >
> >
> > Thanks for the patch! Please see my comments inline.
> >
> > [snip]
> >
>
> Thanks for your comments. Please check my replies inline.
>

Thanks! I'll snip anything I don't have further comments on.

[snip]
> > > +/* META */
> > > +#define REG_META0_VB2_INDEX                0x14dc
> > > +#define REG_META1_VB2_INDEX                0x151c
> >
> > I don't believe these registers are really for VB2 indexes.
> >
>
> MTK P1 ISP HW supports frame header spare registers for each DMA, such
> as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> frame information in these ISP registers. In this case, we save META0
> VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> spare register. These implementation is designed for non-request 3A
> DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> REG_META0_VB2_INDEX for easy understanding.

Unfortunately it's not a good idea to mix hardware concepts with
naming specific to the OS the driver is written for. Better to keep
the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.

> Moreover, if we only need to
> support request mode, we should remove this here.
>
> cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> cmd_params.meta_frame.enabled_dma = dma_port;
> cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
>

Okay, removing sounds good to me. Let's keep the code simple.

[snip]
> > > +
> > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > +
> > > +   /* Sof, done order check */
> > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > +
> > > +           /* Notify IRQ event and enqueue frame */
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > +           isp_dev->current_frame = hw_frame_num;
> >
> > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > event?
> >
>
> This is a another spare register for frame sequence number usage.
> It comes from struct p1_frame_param:frame_seq_no which is sent by
> SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> Is it a better understanding?

I'm sorry, unfortunately it's still not clear to me. Is it the
sequence number of the frame that was just processed and returned to
the kernel or the next frame that is going to be processed from now
on?

>
> Below is our frame request handling in current design.
>
> 1. Buffer preparation
> - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> other meta histogram buffers (LCSO/LMVO) into one request.
> - Accumulated one unique frame sequence number to each request and send
> this request to the SCP composer to compose CQ (Command queue) buffer
> via SCP_ISP_FRAME IPI command.
> - CQ buffer is frame registers set. If ISP registers should be updated
> per frame, these registers are configured in the CQ buffer, such as
> frame sequence number, DMA addresses and tuning ISP registers.
> - One frame request will be composed into one CQ buffer.Once CQ buffer
> is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> its corresponding frame sequence number. Based on this, kernel driver
> knows which request is ready to be en-queued and save this with
> p1_dev->isp_ctx.composed_frame_id.

Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
already have a linked lists of requests that are composed and ready to
be enqueued? Also, the request itself would contain its frame ID
inside the driver request struct, right?

> - The maximum number of CQ buffers in SCP is 3.
>
> 2. Buffer en-queue flow
> - In order to configure correct CQ buffer setting before next SQF event,
> it is depended on by MTK ISP P1 HW CQ mechanism.
> - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> when HW_PASS1_DON_ST is received which means DMA output is done.
> - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> address is used. Otherwise, it will loaded one dummy CQ buffer to
> bypass.
> - So we will check available CQ buffers by comparing composed frame
> sequence number & dequeued frame sequence from ISP HW in SOF event.
> - If there are available CQ buffers, update the CQ base address to the
> next CQ buffer address based on current de-enqueue frame sequence
> number. So MTK ISP P1 HW will load this CQ buffer into HW when
> HW_PASS1_DON_ST is triggered which is before the next SOF.
> - So in next SOF event, ISP HW starts to output DMA buffers with this
> request until request is done.
> - But, for the first request, it is loaded into HW manually when
> streaming is on for better performance.
>
> 3. Buffer de-queue flow
> - We will use frame sequence number to decide which request is ready to
> de-queue.
> - We will save some important register setting from ISP HW when SOF is
> received. This is because the ISP HW starts to output the data with the
> corresponding settings, especially frame sequence number setting.

Could you explain a bit more about these important register settings?
When does the hardware update the values in the register to new ones?
At SOF?

> - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> done. So we could call isp_deque_request_frame with frame sequence
> number to de-queue frame to VB2

What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?

> - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> AAO/AFO non-request buffers are en-queued without request buffers at the
> same time, there will be no SW P1 done event for AAO/AFO DMA done.
> Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.

Do we have a case like this? Wouldn't we normally always want to
bundle AAO/AFO buffers with frame buffers?

> - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> we may try to send another request to SCP for composing.

Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
buffers, right?

>
> Hopefully, my explanation is helpful for better understanding our
> implementation. If you still have any questions, please let me know.
>

Yes, it's more clear now, thanks. Still some more comments above, though.

> > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +   } else {
> > > +           if (irq_status & SOF_INT_ST) {
> > > +                   isp_dev->current_frame = hw_frame_num;
> > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > +           }
> > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > +   }
> >
> > The if and else blocks do almost the same things just in different order. Is
> > it really expected?
> >
>
> If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> any de-queued frame and update the next frame setting later.
> Normally, this is a corner case or system performance issue.

So it sounds like HW_PASS1_DON_ST means that all data from current
frame has been written, right? If I understand your explanation above
correctly, that would mean following handling of each interrupt:

HW_PASS1_DON_ST:
 - CQ executes with next CQ buffer to prepare for next frame. <- how
is this handled? does the CQ hardware automatically receive this event
from the ISP hadware?
 - return VB2 buffers,
 - complete requests.

SOF_INT_ST:
 - send VSYNC event to userspace,
 - program next CQ buffer to CQ,

SW_PASS1_DON_ST:
 - reclaim CQ buffer and enqueue next frame to composing if available

>
> Btw, we will revise the above implementation as below.
>
>
> if (irq_status & SOF_INT_ST)
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
>                                              dequeue_frame_seq_no);
>
> /* Sof, done order check */
> if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
> /* Notify IRQ event and de-enqueue frame */
> irq_handle_notify_event(p1_dev, irq_status, dma_status);

Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?

[snip]
> > > +/* ISP P1 interface functions */
> > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > +{
> > > +   struct device *dev = &cam_dev->pdev->dev;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   int ret;
> > > +
> > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> > > +
> > > +   ret = isp_init_context(p1_dev);
> > > +   if (ret)
> > > +           return ret;
> >
> > The above function doesn't really seem to be related to power management.
> > Should it be called from subdev stream on?
> >
>
> We will rename this function to mtk_isp_hw_init.
> But, it will be called when the first video node is streamed on.
> This is because we need to initialize the HW firstly for sub-device
> stream-on performance.  We need to send some IPI commands, such as
> ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> timing.

What performance do you mean here? The time between first video node
stream on and last video node stream on should be really short. Are
you seeing some long delays there?

That said, doing it when the first video node starts streaming is okay.

[snip]
> > > +   /* Use pure RAW as default HW path */
> > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > +
> > > +   isp_composer_hw_init(dev);
> > > +   /* Check enabled DMAs which is configured by media setup */
> > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> >
> > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > necessary?
> >
>
> Yes, it is necessary for non-request buffers design for Camera 3A video
> nodes. For 3A video nodes, we just want to know which 3A video nodes are
> enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> format from user space. So we just pass the enabled DMA information from
> kernel to SCP only. With 3A enabled DMA information, we could configure
> related 3A registers in SCP.

We should try to remove this non-request mode. Let's continue
discussion on the other patch where I brought this topic.

[snip]
> > > +int mtk_isp_power_release(struct device *dev)
> > > +{
> > > +   isp_composer_hw_deinit(dev);
> > > +   isp_uninit_context(dev);
> >
> > These two don't seem to be related to power either.
> >
> > Instead, I don't see anything that could undo the rproc_boot() operation
> > here.
> >
>
> We will rename this function to mtk_isp_hw_release.
> We will also add rproc_shutdown function call here.
>
> int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> {
>         struct device *dev = cam->dev;
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         isp_composer_hw_deinit(p1_dev);
>         pm_runtime_put_sync_autosuspend(dev);

Note that for autosuspend to work correctly, you also need to call
pm_runtime_mark_last_busy() before this one.

[snip]
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +   struct p1_frame_param frameparams;
> > > +   struct mtk_isp_queue_job *framejob;
> > > +   struct media_request_object *obj, *obj_safe;
> > > +   struct vb2_buffer *vb;
> > > +   struct mtk_cam_dev_buffer *buf;
> > > +
> > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> >
> > This allocation shouldn't be needed. The structure should be already a part
> > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > even say that the contents of this struct should be just moved to that one
> > to avoid overabstracting.
> >
>
> For this function, we will apply the new design from P2 driver's
> comment. Here is the new implementation.
>
> struct mtk_cam_dev_request {
>         struct media_request req;
>         struct mtk_p1_frame_param frame_params;
>         struct work_struct frame_work;
>         struct list_head list;
>         atomic_t buf_count;
> };
>
> void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>                          struct mtk_cam_dev_request *req)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>         int ret;
>
>         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
>         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
>         if (!ret)
>                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
>                         req->frame_params.frame_seq_no, ret);
>         else
>                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>                         req->req.debug_str, req->frame_params.frame_seq_no,
>                         atomic_read(&cam->running_job_count));

It shouldn't be possible for queue_work() to fail here. We just
received a brand new request from the Request API core and called
INIT_WORK() on the work struct.

[snip]
> > > +   enable_sys_clock(p1_dev);
> > > +
> > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > +   if (p1_dev->cam_dev.streaming) {
> > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > +           /* Enable CMOS */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > +           writel((reg_val | CMOS_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > +           /* Enable VF */
> > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > +           writel((reg_val | VFDATA_EN_BIT),
> > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > +   }
> >
> > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > Would the code above continue all the capture from the next buffer, as
> > queued by the userspace before the suspend?
> >
>
> Yes, we will test it.
> 1. SCP buffers are kept by SCP processor
> 2. ISP registers are still kept even if ISP clock is disable.
>

That said, during system suspend, it would be more than just ISP clock
disabled. I'd expect that the ISP power domain would be powered off.
However, if we ensure that the ISP completes before suspend, I guess
that after the resume the next frame CQ buffer would reprogram all the
registers, right?

Also, would SCP always keep running in system suspend?

[snip]
> > > +
> > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> >
> > I think we want to start from 0 here?
> >
>
> Because of single CAM support, we will revise our DTS tree to support
> single CAM only.

Note that DT bindings should describe the hardware not the driver. So
please design the bindings in a way that would cover all the cameras,
even if the driver only takes the information needed to handle 1.

> So we could remove this loop and check the CAM-B HW
> information here. Here is below new function.
>
> static int mtk_isp_probe(struct platform_device *pdev)
> {
>         /* List of clocks required by isp cam */
>         static const char * const clk_names[] = {
>                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
>         };
>         struct mtk_isp_p1_device *p1_dev;
>         struct device *dev = &pdev->dev;
>         struct resource *res;
>         int irq, ret, i;
>
>         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
>         if (!p1_dev)
>                 return -ENOMEM;
>
>         p1_dev->dev = dev;
>         dev_set_drvdata(dev, p1_dev);
>
>         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>         p1_dev->regs = devm_ioremap_resource(dev, res);
>         if (IS_ERR(p1_dev->regs)) {
>                 dev_err(dev, "Failed platform resources map\n");
>                 return PTR_ERR(p1_dev->regs);
>         }
>         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
>
>         irq = platform_get_irq(pdev, 0);
>         if (!irq) {
>                 dev_err(dev, "Missing IRQ resources data\n");
>                 return -ENODEV;
>         }
>         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
>                                dev_name(dev), p1_dev);
>         if (ret) {
>                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
>                         dev->of_node->name, irq);
>                 return ret;
>         }
>         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
>         spin_lock_init(&p1_dev->spinlock_irq);
>
>         p1_dev->num_clks = ARRAY_SIZE(clk_names);
>         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
>                                     sizeof(*p1_dev->clks), GFP_KERNEL);
>         if (!p1_dev->clks)
>                 return -ENOMEM;
>
>         for (i = 0; i < p1_dev->num_clks; ++i)
>                 p1_dev->clks[i].id = clk_names[i];
>
>         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
>                 return ret;
>         }
>
>         ret = isp_setup_scp_rproc(p1_dev, pdev);
>         if (ret)
>                 return ret;
>
>         pm_runtime_enable(dev);

We also need to call pm_runtime_use_autosuspend() and
pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
suggest an autosuspend delay equal to around 2x the time that's needed
to stop and start streaming in total.

[snip]
> > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> >
> > For V4L2 drivers system and runtime PM ops would normally be completely
> > different. Runtime PM ops would be called when the hardware is idle already
> > or is about to become active. System PM ops would be called at system power
> > state change and the hardware might be both idle or active. Please also see
> > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> >
>
> Here is the new implementation. It should be clear to show the
> difference between system and runtime PM ops.
>
> static const struct dev_pm_ops mtk_isp_pm_ops = {
>         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
>                                 pm_runtime_force_resume)
>         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> NULL)
> };

That's still not correct. In runtime suspend/resume ops we already are
not streaming anymore, because we call pm_runtime_get/put_*() when
starting and stopping streaming. In system suspend/resume ops we might
be streaming and that's when we need to stop the hardware and wait for
it to finish. Please implement these ops separately.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
  2019-07-21  2:18           ` Jungo Lin
  (?)
@ 2019-07-25 10:56             ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25 10:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport-F7+t8E8rja9g9hUCZPvPmw, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab

Hi Jungo,

On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
[snip]
> > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > +   }
> > > +
> > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > +   }
> > > +   mutex_unlock(&isp_ctx->lock);
> > > +
> > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> >
> > No need to use the sync variant.
> >
>
> We don't get this point. If we will call pm_runtime_get_sync in
> mtk_isp_hw_init function, will we need to call
> pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> As we know, we should call runtime pm functions in pair.
>

My point is that pm_runtime_put_sync() is only needed if one wants the
runtime count to be decremented after the function returns. Normally
there is no need to do so and one would call pm_runtime_put(), or if
autosuspend is used, pm_runtime_put_autosuspend() (note there is no
"sync" in the name).

[snip]
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > +{
> > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > +   struct device *dev = &p1_dev->pdev->dev;
> > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > +
> > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> >
> > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > contain some extra bytes at the end.)
> >
>
> The len parameter is the actual sending bytes from SCP to kernel.
> In the runtime, it is only 6 bytes for isp_ack_info command
> However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> union structure.
>

That said we still should check if len is enough to cover the data
we're accessing below.

> > > +
> > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > +           return;
> > > +
> > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > +           dev_dbg(dev, "ack frame_num:%d",
> > > +                   ipi_msg->ack_info.frame_seq_no);
> > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > +                      ipi_msg->ack_info.frame_seq_no);
> >
> > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > frames and we should handle that somehow.
> >
>
> No, we use isp_ctx->composed_frame_id to save which frame sequence
> number are composed done in SCP. In new design, we will move this from
> isp_ctx to p1_dev.

But we compose the frames in order, don't we? Wouldn't every composed
frame would be just previous frame ID + 1?

[snip]
> > > +void isp_composer_hw_init(struct device *dev)
> > > +{
> > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +
> > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> >
> > Should we also specify the size of the buffer? Otherwise we could end up
> > with some undetectable overruns.
> >
>
> The size of SCP composer's memory is fixed to 0x200000.
> Is it necessary to specify the size of this buffer?
>
> #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
>
> ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>

Okay, but please add a comment saying that this is an implicit
requirement of the firmware.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-25 10:56             ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25 10:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi Jungo,

On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
[snip]
> > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > +   }
> > > +
> > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > +   }
> > > +   mutex_unlock(&isp_ctx->lock);
> > > +
> > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> >
> > No need to use the sync variant.
> >
>
> We don't get this point. If we will call pm_runtime_get_sync in
> mtk_isp_hw_init function, will we need to call
> pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> As we know, we should call runtime pm functions in pair.
>

My point is that pm_runtime_put_sync() is only needed if one wants the
runtime count to be decremented after the function returns. Normally
there is no need to do so and one would call pm_runtime_put(), or if
autosuspend is used, pm_runtime_put_autosuspend() (note there is no
"sync" in the name).

[snip]
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > +{
> > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > +   struct device *dev = &p1_dev->pdev->dev;
> > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > +
> > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> >
> > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > contain some extra bytes at the end.)
> >
>
> The len parameter is the actual sending bytes from SCP to kernel.
> In the runtime, it is only 6 bytes for isp_ack_info command
> However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> union structure.
>

That said we still should check if len is enough to cover the data
we're accessing below.

> > > +
> > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > +           return;
> > > +
> > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > +           dev_dbg(dev, "ack frame_num:%d",
> > > +                   ipi_msg->ack_info.frame_seq_no);
> > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > +                      ipi_msg->ack_info.frame_seq_no);
> >
> > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > frames and we should handle that somehow.
> >
>
> No, we use isp_ctx->composed_frame_id to save which frame sequence
> number are composed done in SCP. In new design, we will move this from
> isp_ctx to p1_dev.

But we compose the frames in order, don't we? Wouldn't every composed
frame would be just previous frame ID + 1?

[snip]
> > > +void isp_composer_hw_init(struct device *dev)
> > > +{
> > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +
> > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> >
> > Should we also specify the size of the buffer? Otherwise we could end up
> > with some undetectable overruns.
> >
>
> The size of SCP composer's memory is fixed to 0x200000.
> Is it necessary to specify the size of this buffer?
>
> #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
>
> ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>

Okay, but please add a comment saying that this is an implicit
requirement of the firmware.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-25 10:56             ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-25 10:56 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
[snip]
> > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > +   }
> > > +
> > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > +   }
> > > +   mutex_unlock(&isp_ctx->lock);
> > > +
> > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> >
> > No need to use the sync variant.
> >
>
> We don't get this point. If we will call pm_runtime_get_sync in
> mtk_isp_hw_init function, will we need to call
> pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> As we know, we should call runtime pm functions in pair.
>

My point is that pm_runtime_put_sync() is only needed if one wants the
runtime count to be decremented after the function returns. Normally
there is no need to do so and one would call pm_runtime_put(), or if
autosuspend is used, pm_runtime_put_autosuspend() (note there is no
"sync" in the name).

[snip]
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > +{
> > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > +   struct device *dev = &p1_dev->pdev->dev;
> > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > +
> > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> >
> > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > contain some extra bytes at the end.)
> >
>
> The len parameter is the actual sending bytes from SCP to kernel.
> In the runtime, it is only 6 bytes for isp_ack_info command
> However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> union structure.
>

That said we still should check if len is enough to cover the data
we're accessing below.

> > > +
> > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > +           return;
> > > +
> > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > +           dev_dbg(dev, "ack frame_num:%d",
> > > +                   ipi_msg->ack_info.frame_seq_no);
> > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > +                      ipi_msg->ack_info.frame_seq_no);
> >
> > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > frames and we should handle that somehow.
> >
>
> No, we use isp_ctx->composed_frame_id to save which frame sequence
> number are composed done in SCP. In new design, we will move this from
> isp_ctx to p1_dev.

But we compose the frames in order, don't we? Wouldn't every composed
frame would be just previous frame ID + 1?

[snip]
> > > +void isp_composer_hw_init(struct device *dev)
> > > +{
> > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > +
> > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> >
> > Should we also specify the size of the buffer? Otherwise we could end up
> > with some undetectable overruns.
> >
>
> The size of SCP composer's memory is fixed to 0x200000.
> Is it necessary to specify the size of this buffer?
>
> #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
>
> ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>

Okay, but please add a comment saying that this is an implicit
requirement of the firmware.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-23  8:21                     ` Jungo Lin
  (?)
@ 2019-07-26  5:15                       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:15 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab

On Tue, Jul 23, 2019 at 5:22 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi Tomasz:
> > >
> > > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi Tomasz,
> > >
> > > [snip]
> > >
> > > > > After applying your suggestion in SCP device driver, we could remove
> > > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > > SCP processor.
> > > > >
> > > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > > it to our ISP HW device to proceed.
> > > > >
> > > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > > >
> > > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > > >         if (!ptr) {
> > > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > > >                 return -ENOMEM;
> > > > >         }
> > > > >         isp_ctx->scp_mem_pa = addr;
> > > >
> > > > addr contains a DMA address, not a physical address. Could we call it
> > > > scp_mem_dma instead?
> > > >
> > > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > > >
> > > > >         /* get iova address */
> > > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > > >
> > > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > > simplest thing here would be to use dma_map_single() with ptr as the
> > > > CPU address expected.
> > > >
> > >
> > > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > > error. From the debug log of iommu_dma_map_page[3], we got
> > > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > > There is a address offset(0x4000000). If we change to use
> > > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > > expected[2]. Do you have any suggestion on this issue? Do we miss
> > > something?
> >
> > Sorry for the late reply. Could you show me the code changes you made
> > to use dma_map_single()? It would sound like the virtual address
> > passed to dma_map_single() isn't correct.
> >
> > Best regards,
> > Tomasz
> >
>
>
> Please check the below code snippet in today's testing.
>
>         p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /* get iova address */
>         addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
> DMA_BIDIRECTIONAL);
>         if (dma_mapping_error(dev, addr)) {
>                 dma_free_coherent(p1_dev->cam_dev.smem_dev,
>                                   MTK_ISP_COMPOSER_MEM_SIZE,
>                                   ptr, p1_dev->composer_scp_addr);
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, below is extracted log[2].
>
> We guess the virtual address which is returned by dma_alloc_coherent
> function is not valid kernel logical address. It is actually returned by
> memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
> will call virt_to_page() function. For virt_to_page function, it
> requires a logical address[1].
>
> [1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html
>

Indeed virt_to_page() works only with kernel LOWMEM addresses. Whether
virt_to_page() is the right thing to do in dma_map_single() is a good
question, but let's assume it was implemented like this for a reason.

However, you also can't call phys_to_page() on the DMA addresses
returned by dma_alloc_*() either. It works just by luck, because SCP
DMA addresses and CPU physical addresses are numerically the same.

Could you try dma_get_sgtable() with the SCP struct device and then
dma_map_sg() with the P1 struct device?

Best regards,
Tomasz

> [2]
>   322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:00000000a3adc471
>   323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>  7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:0000000042ec580f
>  7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
> 15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:000000005e5b3462
> 15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>
> Best regards,
>
> Jungo
>
> > >
> > > [1]
> > > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > >
> > > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > >
> > > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > > [2]
> > > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > >
> > > [3]
> > > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> > >                 unsigned long offset, size_t size, int prot)
> > > {
> > >         phys_addr_t phys = page_to_phys(page);
> > >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> > >
> > >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> > >                         iommu_get_dma_domain(dev));
> > > }
> > >
> > > [snip]
> > >
> > > Best regards,
> > >
> > > Jungo
> > >
> >
> > _______________________________________________
> > Linux-mediatek mailing list
> > Linux-mediatek@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-mediatek
>
>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  5:15                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:15 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Tue, Jul 23, 2019 at 5:22 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi Tomasz:
> > >
> > > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi Tomasz,
> > >
> > > [snip]
> > >
> > > > > After applying your suggestion in SCP device driver, we could remove
> > > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > > SCP processor.
> > > > >
> > > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > > it to our ISP HW device to proceed.
> > > > >
> > > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > > >
> > > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > > >         if (!ptr) {
> > > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > > >                 return -ENOMEM;
> > > > >         }
> > > > >         isp_ctx->scp_mem_pa = addr;
> > > >
> > > > addr contains a DMA address, not a physical address. Could we call it
> > > > scp_mem_dma instead?
> > > >
> > > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > > >
> > > > >         /* get iova address */
> > > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > > >
> > > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > > simplest thing here would be to use dma_map_single() with ptr as the
> > > > CPU address expected.
> > > >
> > >
> > > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > > error. From the debug log of iommu_dma_map_page[3], we got
> > > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > > There is a address offset(0x4000000). If we change to use
> > > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > > expected[2]. Do you have any suggestion on this issue? Do we miss
> > > something?
> >
> > Sorry for the late reply. Could you show me the code changes you made
> > to use dma_map_single()? It would sound like the virtual address
> > passed to dma_map_single() isn't correct.
> >
> > Best regards,
> > Tomasz
> >
>
>
> Please check the below code snippet in today's testing.
>
>         p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /* get iova address */
>         addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
> DMA_BIDIRECTIONAL);
>         if (dma_mapping_error(dev, addr)) {
>                 dma_free_coherent(p1_dev->cam_dev.smem_dev,
>                                   MTK_ISP_COMPOSER_MEM_SIZE,
>                                   ptr, p1_dev->composer_scp_addr);
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, below is extracted log[2].
>
> We guess the virtual address which is returned by dma_alloc_coherent
> function is not valid kernel logical address. It is actually returned by
> memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
> will call virt_to_page() function. For virt_to_page function, it
> requires a logical address[1].
>
> [1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html
>

Indeed virt_to_page() works only with kernel LOWMEM addresses. Whether
virt_to_page() is the right thing to do in dma_map_single() is a good
question, but let's assume it was implemented like this for a reason.

However, you also can't call phys_to_page() on the DMA addresses
returned by dma_alloc_*() either. It works just by luck, because SCP
DMA addresses and CPU physical addresses are numerically the same.

Could you try dma_get_sgtable() with the SCP struct device and then
dma_map_sg() with the P1 struct device?

Best regards,
Tomasz

> [2]
>   322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:00000000a3adc471
>   323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>  7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:0000000042ec580f
>  7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
> 15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:000000005e5b3462
> 15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>
> Best regards,
>
> Jungo
>
> > >
> > > [1]
> > > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > >
> > > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > >
> > > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > > [2]
> > > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > >
> > > [3]
> > > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> > >                 unsigned long offset, size_t size, int prot)
> > > {
> > >         phys_addr_t phys = page_to_phys(page);
> > >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> > >
> > >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> > >                         iommu_get_dma_domain(dev));
> > > }
> > >
> > > [snip]
> > >
> > > Best regards,
> > >
> > > Jungo
> > >
> >
> > _______________________________________________
> > Linux-mediatek mailing list
> > Linux-mediatek@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-mediatek
>
>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  5:15                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:15 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Tue, Jul 23, 2019 at 5:22 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 16:20 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 5, 2019 at 4:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi Tomasz:
> > >
> > > On Fri, 2019-07-05 at 13:22 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Fri, Jul 5, 2019 at 12:33 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi Tomasz,
> > >
> > > [snip]
> > >
> > > > > After applying your suggestion in SCP device driver, we could remove
> > > > > mtk_cam-smem.h/c. Currently, we use dma_alloc_coherent with SCP device
> > > > > to get SCP address. We could touch the buffer with this SCP address in
> > > > > SCP processor.
> > > > >
> > > > > After that, we use dma_map_page_attrs with P1 device which supports
> > > > > IOMMU domain to get IOVA address. For this address, we will assign
> > > > > it to our ISP HW device to proceed.
> > > > >
> > > > > Below is the snippet for ISP P1 compose buffer initialization.
> > > > >
> > > > >         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > > > >                                  MAX_COMPOSER_SIZE, &addr, GFP_KERNEL);
> > > > >         if (!ptr) {
> > > > >                 dev_err(dev, "failed to allocate compose memory\n");
> > > > >                 return -ENOMEM;
> > > > >         }
> > > > >         isp_ctx->scp_mem_pa = addr;
> > > >
> > > > addr contains a DMA address, not a physical address. Could we call it
> > > > scp_mem_dma instead?
> > > >
> > > > >         dev_dbg(dev, "scp addr:%pad\n", &addr);
> > > > >
> > > > >         /* get iova address */
> > > > >         addr = dma_map_page_attrs(dev, phys_to_page(addr), 0,
> > > >
> > > > addr is a DMA address, so phys_to_page() can't be called on it. The
> > > > simplest thing here would be to use dma_map_single() with ptr as the
> > > > CPU address expected.
> > > >
> > >
> > > We have changed to use ma_map_single() with ptr, but encounter IOMMU
> > > error. From the debug log of iommu_dma_map_page[3], we got
> > > 0x0000000054800000 instead of expected address: 0x0000000050800000[2].
> > > There is a address offset(0x4000000). If we change to use
> > > dma_map_page_attrs with phys_to_page(addr), the address is correct as we
> > > expected[2]. Do you have any suggestion on this issue? Do we miss
> > > something?
> >
> > Sorry for the late reply. Could you show me the code changes you made
> > to use dma_map_single()? It would sound like the virtual address
> > passed to dma_map_single() isn't correct.
> >
> > Best regards,
> > Tomasz
> >
>
>
> Please check the below code snippet in today's testing.
>
>         p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_info(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /* get iova address */
>         addr = dma_map_single(dev, ptr, MTK_ISP_COMPOSER_MEM_SIZE,
> DMA_BIDIRECTIONAL);
>         if (dma_mapping_error(dev, addr)) {
>                 dma_free_coherent(p1_dev->cam_dev.smem_dev,
>                                   MTK_ISP_COMPOSER_MEM_SIZE,
>                                   ptr, p1_dev->composer_scp_addr);
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, below is extracted log[2].
>
> We guess the virtual address which is returned by dma_alloc_coherent
> function is not valid kernel logical address. It is actually returned by
> memremap() in dma_init_coherent_memory(). Moreover, dma_map_single()
> will call virt_to_page() function. For virt_to_page function, it
> requires a logical address[1].
>
> [1]https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch15.html
>

Indeed virt_to_page() works only with kernel LOWMEM addresses. Whether
virt_to_page() is the right thing to do in dma_map_single() is a good
question, but let's assume it was implemented like this for a reason.

However, you also can't call phys_to_page() on the DMA addresses
returned by dma_alloc_*() either. It works just by luck, because SCP
DMA addresses and CPU physical addresses are numerically the same.

Could you try dma_get_sgtable() with the SCP struct device and then
dma_map_sg() with the P1 struct device?

Best regards,
Tomasz

> [2]
>   322 [    1.238269] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:00000000a3adc471
>   323 [    1.239582] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>  7716 [    1.238963] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:0000000042ec580f
>  7717 [    1.240276] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
> 15088 [    1.239309] mtk-cam-p1 1a006000.camisp: scp
> addr:0x0000000052000000 va:000000005e5b3462
> 15089 [    1.240626] mtk-cam-p1 1a006000.camisp: scp iova
> addr:0x00000000fde00000
>
> Best regards,
>
> Jungo
>
> > >
> > > [1]
> > > [    1.344786] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > >
> > > [    1.346890] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > >
> > > [    1.347864] iommu_dma_map_page:0x0000000054800000 offset:0
> > > [    1.348562] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > > [2]
> > > [    1.346738] __dma_alloc_from_coherent: 0x800000 PAGE_SHIFT:12
> > > device_base:0x0000000050000000 dma:0x0000000050800000
> > > virt_base:ffffff8014000000 va:ffffff8014800000
> > > [    1.348841] mtk-cam 1a000000.camisp: scp addr:0x0000000050800000
> > > va:ffffff8014800000
> > > [    1.349816] iommu_dma_map_page:0x0000000050800000 offset:0
> > > [    1.350514] mtk-cam 1a000000.camisp: iova addr:0x00000000fde00000
> > >
> > >
> > > [3]
> > > dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> > >                 unsigned long offset, size_t size, int prot)
> > > {
> > >         phys_addr_t phys = page_to_phys(page);
> > >         pr_err("iommu_dma_map_page:%pa offset:%lu\n", &phys, offset);
> > >
> > >         return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot,
> > >                         iommu_get_dma_domain(dev));
> > > }
> > >
> > > [snip]
> > >
> > > Best regards,
> > >
> > > Jungo
> > >
> >
> > _______________________________________________
> > Linux-mediatek mailing list
> > Linux-mediatek@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-mediatek
>
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-24  4:31             ` Jungo Lin
  (?)
@ 2019-07-26  5:49               ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:49 UTC (permalink / raw)
  To: Jungo Lin, Hans Verkuil
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport-F7+t8E8rja9g9hUCZPvPmw, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list-Y9sIeH5OGRo@public.gmane.org:IOMMU DRIVERS
	<iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org>,
	Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>,

On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > [snip]
> > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > +                                 struct media_request *new_req)
> > > > > +{
> > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +
> > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > +
> > > > > +   if (!cam_dev->streaming) {
> > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Normal enqueue flow */
> > > > > +   if (new_req) {
> > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Flush all media requests wehen first stream on */
> > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > +           list_del(&req->list);
> > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > +   }
> > > > > +}
> > > >
> > > > This will have to be redone, as per the other suggestions, but generally one
> > > > would have a function that tries to queue as much as possible from a list to
> > > > the hardware and another function that adds a request to the list and calls
> > > > the first function.
> > > >
> > >
> > > We revised this function as below.
> > > First to check the en-queue conditions:
> > > a. stream on
> > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > at the same time.
> > >
> > >
> > > Second, try to en-queue the frames in the pending job if possible and
> > > move them into running job list if possible.
> > >
> > > The request has been inserted into pending job in mtk_cam_req_validate
> > > which is used to validate media_request.
> >
> > Thanks for replying to each of the comments, that's very helpful.
> > Snipped out the parts that I agreed with.
> >
> > Please note that req_validate is not supposed to change any driver
> > state. It's only supposed to validate the request. req_queue is the
> > right callback to insert the request into some internal driver
> > bookkeeping structures.
> >
>
> Yes, in req_validate function, we don't change any driver state.
> Below is the function's implementation.
>
> a. Call vb2_request_validate(req) to verify media request.
> b. Update the buffer internal structure buffer.
> c. Insert the request into pending_job_list to prepare en-queue.
>

Adding to a list is changing driver state. The callback must not
modify anything else than the request itself.

Queuing to driver's list should happen in req_queue instead.

[snip]
> > >
> > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > {
> > >         struct mtk_cam_dev_request *req, *req_prev;
> > >         struct list_head enqueue_job_list;
> > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > >         unsigned long flags;
> > >
> > >         if (!cam_dev->streaming ||
> > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> >
> > Do we have a guarantee that cam_dev->running_job_count doesn't
> > decrement between the atomic_read() above and this line?
> >
>
> Ok, we will use cam->pending_job_lock to protect
> cam_dev->running_job_count access. Below is the revised version.
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {

If we use a spin_lock to protect the counter, perhaps we don't need
the atomic type anymore?

>                 dev_dbg(cam->dev, "jobs are full\n");
>                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>                 return;
>         }
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {

Could we instead check the counter here and break if it's >=
MTK_ISP_MAX_RUNNING_JOBS?
Then we could increment it here too to simplify the code.

>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam->running_job_list);
>                 mtk_isp_req_enqueue(cam, req);
>                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
>                 if (atomic_inc_return(&cam->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;

With the above suggestion, this if block would go away.

[snip]
> > >                 mtk_isp_req_enqueue(cam_dev, req);
> > >         }
> > > }
> > >
> > [snip]
> > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > +
> > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > +           stride = ALIGN(stride, 4);
> > > >
> > > > Is it expected that only the F10 format needs this alignment?
> > > >
> > >
> > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > comments.
> >
> > That means that the B10 format also needs the extra alignment, as
> > opposed to what the original code did, right?
> >
>
> Sorry for short code snippet.
> This alignment checking is only applied to F10, no B10.
> If you like to check the full function, you could check this in this
> link[1].
>
> static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> node_id,
>                              struct v4l2_pix_format_mplane *mp)
> {
>         unsigned int bpl, ppl;
>         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>         unsigned int width = mp->width;
>
>         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
>                 /* bayer encoding format & 2 bytes alignment */
>                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
>         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
>                 /*
>                  * The FULL-G encoding format
>                  * 1 G component per pixel
>                  * 1 R component per 4 pixel
>                  * 1 B component per 4 pixel
>                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
>                  */
>                 ppl = DIV_ROUND_UP(width * 6, 4);
>                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
>
>                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
>                 if (pixel_bits == 10)
>                         bpl = ALIGN(bpl, 4);
>                 else
>                         bpl = ALIGN(bpl, 8);
>         }
>
> [1]
> https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
>

Got it, thanks!

[snip]
> > > > > +
> > > > > +static struct v4l2_subdev *
> > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > +{
> > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > +   struct media_entity *entity;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +   struct v4l2_subdev *sensor;
> > > >
> > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > there no compiler warning generated for this?
> > > >
> > >
> > > No, there is no compiler warning.
> > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > with different compiler options.
> > >
> >
> > Thanks. It would be useful if you could check why the compiler you're
> > using doesn't show a warning here. We might be missing other
> > uninitialized variables.
> >
>
> We will feedback to your project team to check the possible reason about
> compiler warning issue.
>

Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
emerge chromeos-kernel-4_19)?

[snip]
> > > > > +
> > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > +           __func__,
> > > > > +           node->id,
> > > > > +           buf->vbb.request_fd,
> > > > > +           buf->vbb.vb2_buf.index);
> > > > > +
> > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > +           return;
> > > >
> > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > with a need to provide compatibility for non-request mode, then it should be
> > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > deal with two modes.
> > > >
> > >
> > > The purpose of non-request function in this driver is needed by
> > > our camera middle-ware design. It needs 3A statistics buffers before
> > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > will en-queue the images, tuning buffer and other meta buffers with
> > > request mode. Based on this requirement, do you have any suggestion?
> > > For upstream driver, should we only consider request mode?
> > >
> >
> > Where does that requirement come from? Why the timing of queuing of
> > the buffers to the driver is important?
> >
> > [snip]
>
> Basically, this requirement comes from our internal camera
> middle-ware/3A hal in user space. Since this is not generic requirement,
> we will follow your original suggestion to keep the request mode only
> and remove other non-request design in other files. For upstream driver,
> it should support request mode only.
>

Note that Chromium OS will use the "upstream driver" and we don't want
to diverge, so please make the userspace also use only requests. I
don't see a reason why there would be any need to submit any buffers
outside of a request.

[snip]
> > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > +{
> > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > +
> > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > >
> > > > This would end up being called multiple times, once for each video node.
> > > > Instead, this should be called explicitly by the driver when it completed
> > > > the request - perhaps in the frame completion handler?
> > > >
> > > > With that, we probably wouldn't even need this callback.
> > > >
> > >
> > > First, if we don't implement this callback function, we will receive
> > > kernel warning as below.
> > >
> > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > >
> > > Second, this function is only be called in __vb2_queue_cancel function.
> > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > So could we just implement dummy empty function?
> > >
> > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > but is
> > >  *                      associated with a queued request was canceled.
> > >  *                      The driver will have to mark associated objects in the
> > >  *                      request as completed; required if requests are
> > >  *                      supported.
> > >
> >
> > Good catch, thanks.
> >
> > Sounds like we may indeed need to implement this callback. In
> > particular, we may need to remove the request that the buffer was
> > associated with from the driver queue and return the other buffers
> > associated to it with an error state. This should be similar to
> > handling a request failure.
> > [snip]
>
> Before calling this callback function, the VB2's stop_streaming has been
> called. Normally, we will return the buffers belonged to this vb2 queu
> with error state. On other hand, only if the state of request is
> MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> __vb2_queue_cancel function. It hints this media request has been
> validated and inserted into our driver's pending_job_list or
> running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> requests from driver's list when streaming is off. Since we have no
> v4l2_ctrl, do we need to do the above things which is already handled in
> mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> only designed for v4l2_ctrl_request_complete usage?

Are you sure that this callback can be only called after
stop_streaming? Also wouldn't that be after stop_streaming only on 1
queue? The other queues could still remain streaming, but we still
have to return corresponding buffers I believe.

Hans, could you clarify what exactly this callback is supposed to do?

>
> static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         dev_dbg(cam->dev, "%s\n", __func__);
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> }
>
> static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> {
>         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>         struct device *dev = cam->dev;
>
>         dev_dbg(dev, "%s node:%d count info:%d", __func__,
>                 node->id, atomic_read(&cam->stream_count));
>
>         mutex_lock(&cam->op_lock);
>         if (atomic_read(&cam->stream_count) == cam->enabled_count)
>                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
>                         dev_err(dev, "failed to stop streaming\n");
>
>         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>
>         /* Check the first node to stream-off */
>         if (!atomic_dec_and_test(&cam->stream_count)) {
>                 mutex_unlock(&cam->op_lock);
>                 return;
>         }
>         mutex_unlock(&cam->op_lock);
>
>         mtk_cam_dev_req_cleanup(cam);
>         media_pipeline_stop(&node->vdev.entity);
> }

[keeping the context for Hans]

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-26  5:49               ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:49 UTC (permalink / raw)
  To: Jungo Lin, Hans Verkuil
  Cc: Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > [snip]
> > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > +                                 struct media_request *new_req)
> > > > > +{
> > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +
> > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > +
> > > > > +   if (!cam_dev->streaming) {
> > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Normal enqueue flow */
> > > > > +   if (new_req) {
> > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Flush all media requests wehen first stream on */
> > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > +           list_del(&req->list);
> > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > +   }
> > > > > +}
> > > >
> > > > This will have to be redone, as per the other suggestions, but generally one
> > > > would have a function that tries to queue as much as possible from a list to
> > > > the hardware and another function that adds a request to the list and calls
> > > > the first function.
> > > >
> > >
> > > We revised this function as below.
> > > First to check the en-queue conditions:
> > > a. stream on
> > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > at the same time.
> > >
> > >
> > > Second, try to en-queue the frames in the pending job if possible and
> > > move them into running job list if possible.
> > >
> > > The request has been inserted into pending job in mtk_cam_req_validate
> > > which is used to validate media_request.
> >
> > Thanks for replying to each of the comments, that's very helpful.
> > Snipped out the parts that I agreed with.
> >
> > Please note that req_validate is not supposed to change any driver
> > state. It's only supposed to validate the request. req_queue is the
> > right callback to insert the request into some internal driver
> > bookkeeping structures.
> >
>
> Yes, in req_validate function, we don't change any driver state.
> Below is the function's implementation.
>
> a. Call vb2_request_validate(req) to verify media request.
> b. Update the buffer internal structure buffer.
> c. Insert the request into pending_job_list to prepare en-queue.
>

Adding to a list is changing driver state. The callback must not
modify anything else than the request itself.

Queuing to driver's list should happen in req_queue instead.

[snip]
> > >
> > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > {
> > >         struct mtk_cam_dev_request *req, *req_prev;
> > >         struct list_head enqueue_job_list;
> > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > >         unsigned long flags;
> > >
> > >         if (!cam_dev->streaming ||
> > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> >
> > Do we have a guarantee that cam_dev->running_job_count doesn't
> > decrement between the atomic_read() above and this line?
> >
>
> Ok, we will use cam->pending_job_lock to protect
> cam_dev->running_job_count access. Below is the revised version.
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {

If we use a spin_lock to protect the counter, perhaps we don't need
the atomic type anymore?

>                 dev_dbg(cam->dev, "jobs are full\n");
>                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>                 return;
>         }
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {

Could we instead check the counter here and break if it's >=
MTK_ISP_MAX_RUNNING_JOBS?
Then we could increment it here too to simplify the code.

>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam->running_job_list);
>                 mtk_isp_req_enqueue(cam, req);
>                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
>                 if (atomic_inc_return(&cam->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;

With the above suggestion, this if block would go away.

[snip]
> > >                 mtk_isp_req_enqueue(cam_dev, req);
> > >         }
> > > }
> > >
> > [snip]
> > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > +
> > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > +           stride = ALIGN(stride, 4);
> > > >
> > > > Is it expected that only the F10 format needs this alignment?
> > > >
> > >
> > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > comments.
> >
> > That means that the B10 format also needs the extra alignment, as
> > opposed to what the original code did, right?
> >
>
> Sorry for short code snippet.
> This alignment checking is only applied to F10, no B10.
> If you like to check the full function, you could check this in this
> link[1].
>
> static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> node_id,
>                              struct v4l2_pix_format_mplane *mp)
> {
>         unsigned int bpl, ppl;
>         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>         unsigned int width = mp->width;
>
>         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
>                 /* bayer encoding format & 2 bytes alignment */
>                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
>         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
>                 /*
>                  * The FULL-G encoding format
>                  * 1 G component per pixel
>                  * 1 R component per 4 pixel
>                  * 1 B component per 4 pixel
>                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
>                  */
>                 ppl = DIV_ROUND_UP(width * 6, 4);
>                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
>
>                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
>                 if (pixel_bits == 10)
>                         bpl = ALIGN(bpl, 4);
>                 else
>                         bpl = ALIGN(bpl, 8);
>         }
>
> [1]
> https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
>

Got it, thanks!

[snip]
> > > > > +
> > > > > +static struct v4l2_subdev *
> > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > +{
> > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > +   struct media_entity *entity;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +   struct v4l2_subdev *sensor;
> > > >
> > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > there no compiler warning generated for this?
> > > >
> > >
> > > No, there is no compiler warning.
> > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > with different compiler options.
> > >
> >
> > Thanks. It would be useful if you could check why the compiler you're
> > using doesn't show a warning here. We might be missing other
> > uninitialized variables.
> >
>
> We will feedback to your project team to check the possible reason about
> compiler warning issue.
>

Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
emerge chromeos-kernel-4_19)?

[snip]
> > > > > +
> > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > +           __func__,
> > > > > +           node->id,
> > > > > +           buf->vbb.request_fd,
> > > > > +           buf->vbb.vb2_buf.index);
> > > > > +
> > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > +           return;
> > > >
> > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > with a need to provide compatibility for non-request mode, then it should be
> > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > deal with two modes.
> > > >
> > >
> > > The purpose of non-request function in this driver is needed by
> > > our camera middle-ware design. It needs 3A statistics buffers before
> > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > will en-queue the images, tuning buffer and other meta buffers with
> > > request mode. Based on this requirement, do you have any suggestion?
> > > For upstream driver, should we only consider request mode?
> > >
> >
> > Where does that requirement come from? Why the timing of queuing of
> > the buffers to the driver is important?
> >
> > [snip]
>
> Basically, this requirement comes from our internal camera
> middle-ware/3A hal in user space. Since this is not generic requirement,
> we will follow your original suggestion to keep the request mode only
> and remove other non-request design in other files. For upstream driver,
> it should support request mode only.
>

Note that Chromium OS will use the "upstream driver" and we don't want
to diverge, so please make the userspace also use only requests. I
don't see a reason why there would be any need to submit any buffers
outside of a request.

[snip]
> > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > +{
> > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > +
> > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > >
> > > > This would end up being called multiple times, once for each video node.
> > > > Instead, this should be called explicitly by the driver when it completed
> > > > the request - perhaps in the frame completion handler?
> > > >
> > > > With that, we probably wouldn't even need this callback.
> > > >
> > >
> > > First, if we don't implement this callback function, we will receive
> > > kernel warning as below.
> > >
> > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > >
> > > Second, this function is only be called in __vb2_queue_cancel function.
> > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > So could we just implement dummy empty function?
> > >
> > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > but is
> > >  *                      associated with a queued request was canceled.
> > >  *                      The driver will have to mark associated objects in the
> > >  *                      request as completed; required if requests are
> > >  *                      supported.
> > >
> >
> > Good catch, thanks.
> >
> > Sounds like we may indeed need to implement this callback. In
> > particular, we may need to remove the request that the buffer was
> > associated with from the driver queue and return the other buffers
> > associated to it with an error state. This should be similar to
> > handling a request failure.
> > [snip]
>
> Before calling this callback function, the VB2's stop_streaming has been
> called. Normally, we will return the buffers belonged to this vb2 queu
> with error state. On other hand, only if the state of request is
> MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> __vb2_queue_cancel function. It hints this media request has been
> validated and inserted into our driver's pending_job_list or
> running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> requests from driver's list when streaming is off. Since we have no
> v4l2_ctrl, do we need to do the above things which is already handled in
> mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> only designed for v4l2_ctrl_request_complete usage?

Are you sure that this callback can be only called after
stop_streaming? Also wouldn't that be after stop_streaming only on 1
queue? The other queues could still remain streaming, but we still
have to return corresponding buffers I believe.

Hans, could you clarify what exactly this callback is supposed to do?

>
> static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         dev_dbg(cam->dev, "%s\n", __func__);
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> }
>
> static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> {
>         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>         struct device *dev = cam->dev;
>
>         dev_dbg(dev, "%s node:%d count info:%d", __func__,
>                 node->id, atomic_read(&cam->stream_count));
>
>         mutex_lock(&cam->op_lock);
>         if (atomic_read(&cam->stream_count) == cam->enabled_count)
>                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
>                         dev_err(dev, "failed to stop streaming\n");
>
>         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>
>         /* Check the first node to stream-off */
>         if (!atomic_dec_and_test(&cam->stream_count)) {
>                 mutex_unlock(&cam->op_lock);
>                 return;
>         }
>         mutex_unlock(&cam->op_lock);
>
>         mtk_cam_dev_req_cleanup(cam);
>         media_pipeline_stop(&node->vdev.entity);
> }

[keeping the context for Hans]

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-26  5:49               ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  5:49 UTC (permalink / raw)
  To: Jungo Lin, Hans Verkuil
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > [snip]
> > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > +                                 struct media_request *new_req)
> > > > > +{
> > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +
> > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > +
> > > > > +   if (!cam_dev->streaming) {
> > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Normal enqueue flow */
> > > > > +   if (new_req) {
> > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > +           return;
> > > > > +   }
> > > > > +
> > > > > +   /* Flush all media requests wehen first stream on */
> > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > +           list_del(&req->list);
> > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > +   }
> > > > > +}
> > > >
> > > > This will have to be redone, as per the other suggestions, but generally one
> > > > would have a function that tries to queue as much as possible from a list to
> > > > the hardware and another function that adds a request to the list and calls
> > > > the first function.
> > > >
> > >
> > > We revised this function as below.
> > > First to check the en-queue conditions:
> > > a. stream on
> > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > at the same time.
> > >
> > >
> > > Second, try to en-queue the frames in the pending job if possible and
> > > move them into running job list if possible.
> > >
> > > The request has been inserted into pending job in mtk_cam_req_validate
> > > which is used to validate media_request.
> >
> > Thanks for replying to each of the comments, that's very helpful.
> > Snipped out the parts that I agreed with.
> >
> > Please note that req_validate is not supposed to change any driver
> > state. It's only supposed to validate the request. req_queue is the
> > right callback to insert the request into some internal driver
> > bookkeeping structures.
> >
>
> Yes, in req_validate function, we don't change any driver state.
> Below is the function's implementation.
>
> a. Call vb2_request_validate(req) to verify media request.
> b. Update the buffer internal structure buffer.
> c. Insert the request into pending_job_list to prepare en-queue.
>

Adding to a list is changing driver state. The callback must not
modify anything else than the request itself.

Queuing to driver's list should happen in req_queue instead.

[snip]
> > >
> > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > {
> > >         struct mtk_cam_dev_request *req, *req_prev;
> > >         struct list_head enqueue_job_list;
> > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > >         unsigned long flags;
> > >
> > >         if (!cam_dev->streaming ||
> > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> >
> > Do we have a guarantee that cam_dev->running_job_count doesn't
> > decrement between the atomic_read() above and this line?
> >
>
> Ok, we will use cam->pending_job_lock to protect
> cam_dev->running_job_count access. Below is the revised version.
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {

If we use a spin_lock to protect the counter, perhaps we don't need
the atomic type anymore?

>                 dev_dbg(cam->dev, "jobs are full\n");
>                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>                 return;
>         }
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {

Could we instead check the counter here and break if it's >=
MTK_ISP_MAX_RUNNING_JOBS?
Then we could increment it here too to simplify the code.

>                 list_del(&req->list);
>                 spin_lock_irqsave(&cam->running_job_lock, flags);
>                 list_add_tail(&req->list, &cam->running_job_list);
>                 mtk_isp_req_enqueue(cam, req);
>                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
>                 if (atomic_inc_return(&cam->running_job_count) >=
>                         MTK_ISP_MAX_RUNNING_JOBS)
>                         break;

With the above suggestion, this if block would go away.

[snip]
> > >                 mtk_isp_req_enqueue(cam_dev, req);
> > >         }
> > > }
> > >
> > [snip]
> > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > +
> > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > +           stride = ALIGN(stride, 4);
> > > >
> > > > Is it expected that only the F10 format needs this alignment?
> > > >
> > >
> > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > comments.
> >
> > That means that the B10 format also needs the extra alignment, as
> > opposed to what the original code did, right?
> >
>
> Sorry for short code snippet.
> This alignment checking is only applied to F10, no B10.
> If you like to check the full function, you could check this in this
> link[1].
>
> static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> node_id,
>                              struct v4l2_pix_format_mplane *mp)
> {
>         unsigned int bpl, ppl;
>         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>         unsigned int width = mp->width;
>
>         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
>                 /* bayer encoding format & 2 bytes alignment */
>                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
>         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
>                 /*
>                  * The FULL-G encoding format
>                  * 1 G component per pixel
>                  * 1 R component per 4 pixel
>                  * 1 B component per 4 pixel
>                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
>                  */
>                 ppl = DIV_ROUND_UP(width * 6, 4);
>                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
>
>                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
>                 if (pixel_bits == 10)
>                         bpl = ALIGN(bpl, 4);
>                 else
>                         bpl = ALIGN(bpl, 8);
>         }
>
> [1]
> https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
>

Got it, thanks!

[snip]
> > > > > +
> > > > > +static struct v4l2_subdev *
> > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > +{
> > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > +   struct media_entity *entity;
> > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > +   struct v4l2_subdev *sensor;
> > > >
> > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > there no compiler warning generated for this?
> > > >
> > >
> > > No, there is no compiler warning.
> > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > with different compiler options.
> > >
> >
> > Thanks. It would be useful if you could check why the compiler you're
> > using doesn't show a warning here. We might be missing other
> > uninitialized variables.
> >
>
> We will feedback to your project team to check the possible reason about
> compiler warning issue.
>

Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
emerge chromeos-kernel-4_19)?

[snip]
> > > > > +
> > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > +           __func__,
> > > > > +           node->id,
> > > > > +           buf->vbb.request_fd,
> > > > > +           buf->vbb.vb2_buf.index);
> > > > > +
> > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > +           return;
> > > >
> > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > with a need to provide compatibility for non-request mode, then it should be
> > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > deal with two modes.
> > > >
> > >
> > > The purpose of non-request function in this driver is needed by
> > > our camera middle-ware design. It needs 3A statistics buffers before
> > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > will en-queue the images, tuning buffer and other meta buffers with
> > > request mode. Based on this requirement, do you have any suggestion?
> > > For upstream driver, should we only consider request mode?
> > >
> >
> > Where does that requirement come from? Why the timing of queuing of
> > the buffers to the driver is important?
> >
> > [snip]
>
> Basically, this requirement comes from our internal camera
> middle-ware/3A hal in user space. Since this is not generic requirement,
> we will follow your original suggestion to keep the request mode only
> and remove other non-request design in other files. For upstream driver,
> it should support request mode only.
>

Note that Chromium OS will use the "upstream driver" and we don't want
to diverge, so please make the userspace also use only requests. I
don't see a reason why there would be any need to submit any buffers
outside of a request.

[snip]
> > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > +{
> > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > +
> > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > >
> > > > This would end up being called multiple times, once for each video node.
> > > > Instead, this should be called explicitly by the driver when it completed
> > > > the request - perhaps in the frame completion handler?
> > > >
> > > > With that, we probably wouldn't even need this callback.
> > > >
> > >
> > > First, if we don't implement this callback function, we will receive
> > > kernel warning as below.
> > >
> > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > >
> > > Second, this function is only be called in __vb2_queue_cancel function.
> > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > So could we just implement dummy empty function?
> > >
> > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > but is
> > >  *                      associated with a queued request was canceled.
> > >  *                      The driver will have to mark associated objects in the
> > >  *                      request as completed; required if requests are
> > >  *                      supported.
> > >
> >
> > Good catch, thanks.
> >
> > Sounds like we may indeed need to implement this callback. In
> > particular, we may need to remove the request that the buffer was
> > associated with from the driver queue and return the other buffers
> > associated to it with an error state. This should be similar to
> > handling a request failure.
> > [snip]
>
> Before calling this callback function, the VB2's stop_streaming has been
> called. Normally, we will return the buffers belonged to this vb2 queu
> with error state. On other hand, only if the state of request is
> MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> __vb2_queue_cancel function. It hints this media request has been
> validated and inserted into our driver's pending_job_list or
> running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> requests from driver's list when streaming is off. Since we have no
> v4l2_ctrl, do we need to do the above things which is already handled in
> mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> only designed for v4l2_ctrl_request_complete usage?

Are you sure that this callback can be only called after
stop_streaming? Also wouldn't that be after stop_streaming only on 1
queue? The other queues could still remain streaming, but we still
have to return corresponding buffers I believe.

Hans, could you clarify what exactly this callback is supposed to do?

>
> static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         dev_dbg(cam->dev, "%s\n", __func__);
>
>         spin_lock_irqsave(&cam->pending_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
>                 list_del(&req->list);
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> }
>
> static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> {
>         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>         struct device *dev = cam->dev;
>
>         dev_dbg(dev, "%s node:%d count info:%d", __func__,
>                 node->id, atomic_read(&cam->stream_count));
>
>         mutex_lock(&cam->op_lock);
>         if (atomic_read(&cam->stream_count) == cam->enabled_count)
>                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
>                         dev_err(dev, "failed to stop streaming\n");
>
>         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>
>         /* Check the first node to stream-off */
>         if (!atomic_dec_and_test(&cam->stream_count)) {
>                 mutex_unlock(&cam->op_lock);
>                 return;
>         }
>         mutex_unlock(&cam->op_lock);
>
>         mtk_cam_dev_req_cleanup(cam);
>         media_pipeline_stop(&node->vdev.entity);
> }

[keeping the context for Hans]

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-07-25  9:23           ` Tomasz Figa
  (?)
@ 2019-07-26  7:23             ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  7:23 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab, list

Hi, Tomasz:

On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> .Hi Jungo,
> 
> On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > > It handles the ISP HW configuration, provides interrupt handling and
> > > > initializes the V4L2 device nodes and other functions.
> > > >
> > > > (The current metadata interface used in meta input and partial
> > > > meta nodes is only a temporary solution to kick off the driver
> > > > development and is not ready to be reviewed yet.)
> > > >
> > > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > > ---
> > > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > > >  4 files changed, 1457 insertions(+)
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > > >
> > >
> > > Thanks for the patch! Please see my comments inline.
> > >
> > > [snip]
> > >
> >
> > Thanks for your comments. Please check my replies inline.
> >
> 
> Thanks! I'll snip anything I don't have further comments on.
> 
> [snip]

Ok, got it.

> > > > +/* META */
> > > > +#define REG_META0_VB2_INDEX                0x14dc
> > > > +#define REG_META1_VB2_INDEX                0x151c
> > >
> > > I don't believe these registers are really for VB2 indexes.
> > >
> >
> > MTK P1 ISP HW supports frame header spare registers for each DMA, such
> > as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> > frame information in these ISP registers. In this case, we save META0
> > VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> > spare register. These implementation is designed for non-request 3A
> > DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> > mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> > REG_META0_VB2_INDEX for easy understanding.
> 
> Unfortunately it's not a good idea to mix hardware concepts with
> naming specific to the OS the driver is written for. Better to keep
> the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.
> 

Ok, got your point. We will pay attention in next time.
Moreover, we will remove AAO/AFO non-request design in next patch.
So these codes will also be removed.

> > Moreover, if we only need to
> > support request mode, we should remove this here.
> >
> > cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> > cmd_params.meta_frame.enabled_dma = dma_port;
> > cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> > cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> >
> 
> Okay, removing sounds good to me. Let's keep the code simple.
> 
> [snip]

Thanks.

> > > > +
> > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > +
> > > > +   /* Sof, done order check */
> > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > +
> > > > +           /* Notify IRQ event and enqueue frame */
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > +           isp_dev->current_frame = hw_frame_num;
> > >
> > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > event?
> > >
> >
> > This is a another spare register for frame sequence number usage.
> > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > Is it a better understanding?
> 
> I'm sorry, unfortunately it's still not clear to me. Is it the
> sequence number of the frame that was just processed and returned to
> the kernel or the next frame that is going to be processed from now
> on?
> 

It is the next frame that is going to be proceed. 
We simplify the implementation of isp_irq_cam function. The hw_frame_num
is renamed to dequeue_frame_seq_no and saved this value from HW at
SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
next frame to be processed. If there is SW_PASS1_DON_ST, it means this
frame is processed done. We use this value to de-queue the frame request
and return buffers to VB2.

The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
HW_PASS1_DON_ST.

a. SW_PASS_DON_ST is designed for DMAs done event.
If there is no available DMA buffers en-queued into HW, there is no
SW_PADD_DON_ST.

b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
It is paired with SOF IRQ event, even if there is no available DMA
buffers.

static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
			       unsigned int dequeue_frame_seq_no)
{
	dma_addr_t base_addr = p1_dev->composer_iova;
	int composed_frame_seq_no =
atomic_read(&p1_dev->composed_frame_seq_no);
	unsigned int addr_offset;

	/* Send V4L2_EVENT_FRAME_SYNC event */
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
	
	p1_dev->sof_count += 1;
	/* Save dequeue frame information */
	p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;

	/* Update CQ base address if needed */
	if (composed_frame_seq_no <= dequeue_frame_seq_no) {
		dev_dbg(p1_dev->dev,
			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
			composed_frame_seq_no, dequeue_frame_seq_no);
		return;
	}
	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
		(dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
	dev_dbg(p1_dev->dev,
		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
		composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
}

void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
				   unsigned int frame_seq_no)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
			req->frame_params.frame_seq_no, frame_seq_no);

		/* Match by the en-queued request number */
		if (req->frame_params.frame_seq_no == frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
			list_del(&req->list);
			break;
		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space for frame drop */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
			dev_warn(cam->dev, "frame_seq:%d drop\n",
				 req->frame_params.frame_seq_no);
			list_del(&req->list);
		} else {
			break;
		}
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);

static irqreturn_t isp_irq_cam(int irq, void *data)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
	struct device *dev = p1_dev->dev;
	unsigned int dequeue_frame_seq_no;
	unsigned int irq_status, err_status, dma_status;
	unsigned long flags;

	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
	err_status = irq_status & INT_ST_MASK_CAM_ERR;
	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
	dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);

	/*
	 * In normal case, the next SOF ISR should come after HW PASS1 DONE
ISR.
	 * If these two ISRs come together, print warning msg to hint.
	 */
	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

	/* De-queue frame */
	if (irq_status & SW_PASS1_DON_ST) {
		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
					      dequeue_frame_seq_no);
		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
	}

	/* Save frame info. & update CQ address for frame HW en-queue */
	if (irq_status & SOF_INT_ST)
		isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);

	/* Check ISP error status */
	if (err_status) {
		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
		/* Show DMA errors in detail */
		if (err_status & DMA_ERR_ST)
			isp_irq_handle_dma_err(p1_dev);
	}

	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
		p1_dev->sof_count, irq_status, dma_status,
		dequeue_frame_seq_no);

	return IRQ_HANDLED;
}

> >
> > Below is our frame request handling in current design.
> >
> > 1. Buffer preparation
> > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > other meta histogram buffers (LCSO/LMVO) into one request.
> > - Accumulated one unique frame sequence number to each request and send
> > this request to the SCP composer to compose CQ (Command queue) buffer
> > via SCP_ISP_FRAME IPI command.
> > - CQ buffer is frame registers set. If ISP registers should be updated
> > per frame, these registers are configured in the CQ buffer, such as
> > frame sequence number, DMA addresses and tuning ISP registers.
> > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > its corresponding frame sequence number. Based on this, kernel driver
> > knows which request is ready to be en-queued and save this with
> > p1_dev->isp_ctx.composed_frame_id.
> 
> Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> already have a linked lists of requests that are composed and ready to
> be enqueued? Also, the request itself would contain its frame ID
> inside the driver request struct, right?
> 

Below is current implementation for frame request en-queued.
Before en-queued into HW by CQ, the request should be composed by SCP
composer.

a. mtk_cam_dev_req_try_queue()
- Insert the request into p1_dev->running_job_list
b. mtk_isp_req_enqueue()
- Assign new next frame ID to this request.
- Sending to SCP by workqueue
- This request is ready to compose
c. isp_tx_frame_worker()
- Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
- SCP composer will compose the buffer CQ for this request frame based
on struct mtk_p1_frame_param which includes frame ID.
- If scp_ipi_send() is returned, it means the request is composed done.
Or
d. isp_composer_handler()
- If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
- The request is composed done here.
e. isp_irq_handle_sof()
- In SOF timing, we will check there is any available composed CQ
buffers by comparing composed & current de-queued frame ID.

For p1_dev->running_job_list, we can't guarantee the requests are
composed until the end of step c. For step e, we need to know how many
available composed requests are ready to en-queued.

Do you suggest we add another new link-list to save these requests in
step c or we could update p1_dev->composed_frame_seq_no in step c and
remove the implementation in step d[1]?

[1]. isp_composer_handler() is mandatory callback function for SCP
sending API with sync mode design.

static void isp_composer_handler(void *data, unsigned int len, void
*priv)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
	struct mtk_isp_scp_p1_cmd *ipi_msg;

	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

	if (ipi_msg->cmd_id != ISP_CMD_ACK)
		return;

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}
}

> > - The maximum number of CQ buffers in SCP is 3.
> >
> > 2. Buffer en-queue flow
> > - In order to configure correct CQ buffer setting before next SQF event,
> > it is depended on by MTK ISP P1 HW CQ mechanism.
> > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > when HW_PASS1_DON_ST is received which means DMA output is done.
> > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > bypass.
> > - So we will check available CQ buffers by comparing composed frame
> > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > - If there are available CQ buffers, update the CQ base address to the
> > next CQ buffer address based on current de-enqueue frame sequence
> > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > request until request is done.
> > - But, for the first request, it is loaded into HW manually when
> > streaming is on for better performance.
> >
> > 3. Buffer de-queue flow
> > - We will use frame sequence number to decide which request is ready to
> > de-queue.
> > - We will save some important register setting from ISP HW when SOF is
> > received. This is because the ISP HW starts to output the data with the
> > corresponding settings, especially frame sequence number setting.
> 
> Could you explain a bit more about these important register settings?
> When does the hardware update the values in the register to new ones?
> At SOF?
> 

Sorry about my words.
In the current implementation, we just save frame ID.


> > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > done. So we could call isp_deque_request_frame with frame sequence
> > number to de-queue frame to VB2
> 
> What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> 

This is explained above.

> > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > AAO/AFO non-request buffers are en-queued without request buffers at the
> > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> 
> Do we have a case like this? Wouldn't we normally always want to
> bundle AAO/AFO buffers with frame buffers?
> 

For upstream driver, we will remove non-request design.

> > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > we may try to send another request to SCP for composing.
> 
> Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> buffers, right?
> 

We expected the the life cycle of CQ buffer is same as frame request.
So SW_PASS1_DON_ST is good timing to re-queue the next request to
compose.
For the CQ operations, we will explain later.

> >
> > Hopefully, my explanation is helpful for better understanding our
> > implementation. If you still have any questions, please let me know.
> >
> 
> Yes, it's more clear now, thanks. Still some more comments above, though.
> 
> > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +   } else {
> > > > +           if (irq_status & SOF_INT_ST) {
> > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +           }
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > +   }
> > >
> > > The if and else blocks do almost the same things just in different order. Is
> > > it really expected?
> > >
> >
> > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > any de-queued frame and update the next frame setting later.
> > Normally, this is a corner case or system performance issue.
> 
> So it sounds like HW_PASS1_DON_ST means that all data from current
> frame has been written, right? If I understand your explanation above
> correctly, that would mean following handling of each interrupt:
> 
> HW_PASS1_DON_ST:
>  - CQ executes with next CQ buffer to prepare for next frame. <- how
> is this handled? does the CQ hardware automatically receive this event
> from the ISP hadware?
>  - return VB2 buffers,
>  - complete requests.
> 
> SOF_INT_ST:
>  - send VSYNC event to userspace,
>  - program next CQ buffer to CQ,
> 
> SW_PASS1_DON_ST:
>  - reclaim CQ buffer and enqueue next frame to composing if available
> 

Sorry for our implementation of HW_PASS1_DON_ST.
It is confusing. 
Below is the revised version based on your conclusion.
So in our new implemmenation, we just handle SOF_INT_ST &
SW_PASS1_DON_ST events. We just add one warning message for
HW_PASS1_DON_ST
 
HW_PASS1_DON_ST:
- CQ executes with next CQ buffer to prepare for next frame.
 
SOF_INT_ST:
- send VSYNC event to userspace,
- program next CQ buffer to CQ,
 
SW_PASS1_DON_ST:
- reclaim CQ buffer and enqueue next frame to composing if available
- return VB2 buffers,
- complete requests.

For CQ HW operations, it is listed below:

a. The CQ buffer has two kinds of information
 - Which ISP registers needs to be updated.
 - Where the corresponding ISP register data to be read.
b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
event periodically. 
 - Normally, if the ISP HW receives the completed frame and it will
trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
-  So the CQ buffer loading is performed by ISP HW automatically.
c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
to decide which CQ buffer is loaded.
   - So we configure the next CQ base address in SOF.
d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
and update the ISP register values into HW.
   - SCP composer will compose one dummy CQ buffer and assign it to
REG_CQ_THR0_BASEADDR of each CQ buffer.
   - Dummy CQ buffer has no updated ISP registers comparing with other
CQ buffers.
   - With this design, if there is no updated new CQ buffer by driver
which may be caused no en-queue frames from user space. The CQ HW will
load dummy CQ buffer and do nothing.
f. The CQ buffer loading is guaranteed by HW to finish before the next
SOF.

> >
> > Btw, we will revise the above implementation as below.
> >
> >
> > if (irq_status & SOF_INT_ST)
> >         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> >                                              dequeue_frame_seq_no);
> >
> > /* Sof, done order check */
> > if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> >         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> >
> > /* Notify IRQ event and de-enqueue frame */
> > irq_handle_notify_event(p1_dev, irq_status, dma_status);
> 
> Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?
> 
> [snip]

Yes, in the new version, we will add SW_PASS1_DON_ST check before
calling mtk_cam_dev_dequeue_req_frame function.

> > > > +/* ISP P1 interface functions */
> > > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   int ret;
> > > > +
> > > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > > > +
> > > > +   ret = isp_init_context(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > >
> > > The above function doesn't really seem to be related to power management.
> > > Should it be called from subdev stream on?
> > >
> >
> > We will rename this function to mtk_isp_hw_init.
> > But, it will be called when the first video node is streamed on.
> > This is because we need to initialize the HW firstly for sub-device
> > stream-on performance.  We need to send some IPI commands, such as
> > ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> > timing.
> 
> What performance do you mean here? The time between first video node
> stream on and last video node stream on should be really short. Are
> you seeing some long delays there?
> 
> That said, doing it when the first video node starts streaming is okay.
> 
> [snip]

Ok, for IPI command sending performance issue is gone with removing the
design of non-request mode of 3A buffers. We could skip this question.

> > > > +   /* Use pure RAW as default HW path */
> > > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > > +
> > > > +   isp_composer_hw_init(dev);
> > > > +   /* Check enabled DMAs which is configured by media setup */
> > > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> > >
> > > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > > necessary?
> > >
> >
> > Yes, it is necessary for non-request buffers design for Camera 3A video
> > nodes. For 3A video nodes, we just want to know which 3A video nodes are
> > enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> > format from user space. So we just pass the enabled DMA information from
> > kernel to SCP only. With 3A enabled DMA information, we could configure
> > related 3A registers in SCP.
> 
> We should try to remove this non-request mode. Let's continue
> discussion on the other patch where I brought this topic.
> 
> [snip]

Ok. We will remove the non-request in next patch.

> > > > +int mtk_isp_power_release(struct device *dev)
> > > > +{
> > > > +   isp_composer_hw_deinit(dev);
> > > > +   isp_uninit_context(dev);
> > >
> > > These two don't seem to be related to power either.
> > >
> > > Instead, I don't see anything that could undo the rproc_boot() operation
> > > here.
> > >
> >
> > We will rename this function to mtk_isp_hw_release.
> > We will also add rproc_shutdown function call here.
> >
> > int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > {
> >         struct device *dev = cam->dev;
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         isp_composer_hw_deinit(p1_dev);
> >         pm_runtime_put_sync_autosuspend(dev);
> 
> Note that for autosuspend to work correctly, you also need to call
> pm_runtime_mark_last_busy() before this one.
> 
> [snip]

Ok, thanks for your hint.

> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   struct p1_frame_param frameparams;
> > > > +   struct mtk_isp_queue_job *framejob;
> > > > +   struct media_request_object *obj, *obj_safe;
> > > > +   struct vb2_buffer *vb;
> > > > +   struct mtk_cam_dev_buffer *buf;
> > > > +
> > > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > >
> > > This allocation shouldn't be needed. The structure should be already a part
> > > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > > even say that the contents of this struct should be just moved to that one
> > > to avoid overabstracting.
> > >
> >
> > For this function, we will apply the new design from P2 driver's
> > comment. Here is the new implementation.
> >
> > struct mtk_cam_dev_request {
> >         struct media_request req;
> >         struct mtk_p1_frame_param frame_params;
> >         struct work_struct frame_work;
> >         struct list_head list;
> >         atomic_t buf_count;
> > };
> >
> > void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >                          struct mtk_cam_dev_request *req)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >         int ret;
> >
> >         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
> >         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
> >         if (!ret)
> >                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
> >                         req->frame_params.frame_seq_no, ret);
> >         else
> >                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >                         req->req.debug_str, req->frame_params.frame_seq_no,
> >                         atomic_read(&cam->running_job_count));
> 
> It shouldn't be possible for queue_work() to fail here. We just
> received a brand new request from the Request API core and called
> INIT_WORK() on the work struct.
> 
> [snip]

Ok, got it. We will remove this unnecessary error checking.

> > > > +   enable_sys_clock(p1_dev);
> > > > +
> > > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > > +   if (p1_dev->cam_dev.streaming) {
> > > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > > +           /* Enable CMOS */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           writel((reg_val | CMOS_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           /* Enable VF */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > > +           writel((reg_val | VFDATA_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > > +   }
> > >
> > > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > > Would the code above continue all the capture from the next buffer, as
> > > queued by the userspace before the suspend?
> > >
> >
> > Yes, we will test it.
> > 1. SCP buffers are kept by SCP processor
> > 2. ISP registers are still kept even if ISP clock is disable.
> >
> 
> That said, during system suspend, it would be more than just ISP clock
> disabled. I'd expect that the ISP power domain would be powered off.
> However, if we ensure that the ISP completes before suspend, I guess
> that after the resume the next frame CQ buffer would reprogram all the
> registers, right?
> 
> Also, would SCP always keep running in system suspend?
> 
> [snip]

Q1. Firstly, we will make sure the ISP complete the current frame before
suspend. For the resume behavior, we will check internally.

Q2. Sorry for my wrong reply. The CQ buffer data should be kept in
memory, not SCP. SCP should also be stopped when system is suspend.

> > > > +
> > > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > >
> > > I think we want to start from 0 here?
> > >
> >
> > Because of single CAM support, we will revise our DTS tree to support
> > single CAM only.
> 
> Note that DT bindings should describe the hardware not the driver. So
> please design the bindings in a way that would cover all the cameras,
> even if the driver only takes the information needed to handle 1.
> 

Ok, we will cover all Camera ISP P1 HW units in the DT.

> > So we could remove this loop and check the CAM-B HW
> > information here. Here is below new function.
> >
> > static int mtk_isp_probe(struct platform_device *pdev)
> > {
> >         /* List of clocks required by isp cam */
> >         static const char * const clk_names[] = {
> >                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> >         };
> >         struct mtk_isp_p1_device *p1_dev;
> >         struct device *dev = &pdev->dev;
> >         struct resource *res;
> >         int irq, ret, i;
> >
> >         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> >         if (!p1_dev)
> >                 return -ENOMEM;
> >
> >         p1_dev->dev = dev;
> >         dev_set_drvdata(dev, p1_dev);
> >
> >         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >         p1_dev->regs = devm_ioremap_resource(dev, res);
> >         if (IS_ERR(p1_dev->regs)) {
> >                 dev_err(dev, "Failed platform resources map\n");
> >                 return PTR_ERR(p1_dev->regs);
> >         }
> >         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> >
> >         irq = platform_get_irq(pdev, 0);
> >         if (!irq) {
> >                 dev_err(dev, "Missing IRQ resources data\n");
> >                 return -ENODEV;
> >         }
> >         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
> >                                dev_name(dev), p1_dev);
> >         if (ret) {
> >                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
> >                         dev->of_node->name, irq);
> >                 return ret;
> >         }
> >         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
> >         spin_lock_init(&p1_dev->spinlock_irq);
> >
> >         p1_dev->num_clks = ARRAY_SIZE(clk_names);
> >         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> >                                     sizeof(*p1_dev->clks), GFP_KERNEL);
> >         if (!p1_dev->clks)
> >                 return -ENOMEM;
> >
> >         for (i = 0; i < p1_dev->num_clks; ++i)
> >                 p1_dev->clks[i].id = clk_names[i];
> >
> >         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> >         if (ret) {
> >                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> >                 return ret;
> >         }
> >
> >         ret = isp_setup_scp_rproc(p1_dev, pdev);
> >         if (ret)
> >                 return ret;
> >
> >         pm_runtime_enable(dev);
> 
> We also need to call pm_runtime_use_autosuspend() and
> pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
> suggest an autosuspend delay equal to around 2x the time that's needed
> to stop and start streaming in total.
> 
> [snip]

O, we will add these function calls.

> > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > >
> > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > different. Runtime PM ops would be called when the hardware is idle already
> > > or is about to become active. System PM ops would be called at system power
> > > state change and the hardware might be both idle or active. Please also see
> > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > >
> >
> > Here is the new implementation. It should be clear to show the
> > difference between system and runtime PM ops.
> >
> > static const struct dev_pm_ops mtk_isp_pm_ops = {
> >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> >                                 pm_runtime_force_resume)
> >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > NULL)
> > };
> 
> That's still not correct. In runtime suspend/resume ops we already are
> not streaming anymore, because we call pm_runtime_get/put_*() when
> starting and stopping streaming. In system suspend/resume ops we might
> be streaming and that's when we need to stop the hardware and wait for
> it to finish. Please implement these ops separately.
> 
> Best regards,
> Tomasz


Ok, got your point.
Below is the new implementation for your review.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Check ISP is streaming or not */
	if (!p1_dev->cam_dev.streaming)
		goto done;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "Cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

done:
	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Force ISP HW to resume if needed */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	if (!p1_dev->cam_dev.streaming)
		return 0;

	/* Enable CMOS */
	dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "enable clock\n");
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
			   NULL)
};

Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-26  7:23             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  7:23 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi, Tomasz:

On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> .Hi Jungo,
> 
> On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > > It handles the ISP HW configuration, provides interrupt handling and
> > > > initializes the V4L2 device nodes and other functions.
> > > >
> > > > (The current metadata interface used in meta input and partial
> > > > meta nodes is only a temporary solution to kick off the driver
> > > > development and is not ready to be reviewed yet.)
> > > >
> > > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > > ---
> > > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > > >  4 files changed, 1457 insertions(+)
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > > >
> > >
> > > Thanks for the patch! Please see my comments inline.
> > >
> > > [snip]
> > >
> >
> > Thanks for your comments. Please check my replies inline.
> >
> 
> Thanks! I'll snip anything I don't have further comments on.
> 
> [snip]

Ok, got it.

> > > > +/* META */
> > > > +#define REG_META0_VB2_INDEX                0x14dc
> > > > +#define REG_META1_VB2_INDEX                0x151c
> > >
> > > I don't believe these registers are really for VB2 indexes.
> > >
> >
> > MTK P1 ISP HW supports frame header spare registers for each DMA, such
> > as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> > frame information in these ISP registers. In this case, we save META0
> > VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> > spare register. These implementation is designed for non-request 3A
> > DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> > mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> > REG_META0_VB2_INDEX for easy understanding.
> 
> Unfortunately it's not a good idea to mix hardware concepts with
> naming specific to the OS the driver is written for. Better to keep
> the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.
> 

Ok, got your point. We will pay attention in next time.
Moreover, we will remove AAO/AFO non-request design in next patch.
So these codes will also be removed.

> > Moreover, if we only need to
> > support request mode, we should remove this here.
> >
> > cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> > cmd_params.meta_frame.enabled_dma = dma_port;
> > cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> > cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> >
> 
> Okay, removing sounds good to me. Let's keep the code simple.
> 
> [snip]

Thanks.

> > > > +
> > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > +
> > > > +   /* Sof, done order check */
> > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > +
> > > > +           /* Notify IRQ event and enqueue frame */
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > +           isp_dev->current_frame = hw_frame_num;
> > >
> > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > event?
> > >
> >
> > This is a another spare register for frame sequence number usage.
> > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > Is it a better understanding?
> 
> I'm sorry, unfortunately it's still not clear to me. Is it the
> sequence number of the frame that was just processed and returned to
> the kernel or the next frame that is going to be processed from now
> on?
> 

It is the next frame that is going to be proceed. 
We simplify the implementation of isp_irq_cam function. The hw_frame_num
is renamed to dequeue_frame_seq_no and saved this value from HW at
SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
next frame to be processed. If there is SW_PASS1_DON_ST, it means this
frame is processed done. We use this value to de-queue the frame request
and return buffers to VB2.

The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
HW_PASS1_DON_ST.

a. SW_PASS_DON_ST is designed for DMAs done event.
If there is no available DMA buffers en-queued into HW, there is no
SW_PADD_DON_ST.

b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
It is paired with SOF IRQ event, even if there is no available DMA
buffers.

static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
			       unsigned int dequeue_frame_seq_no)
{
	dma_addr_t base_addr = p1_dev->composer_iova;
	int composed_frame_seq_no =
atomic_read(&p1_dev->composed_frame_seq_no);
	unsigned int addr_offset;

	/* Send V4L2_EVENT_FRAME_SYNC event */
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
	
	p1_dev->sof_count += 1;
	/* Save dequeue frame information */
	p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;

	/* Update CQ base address if needed */
	if (composed_frame_seq_no <= dequeue_frame_seq_no) {
		dev_dbg(p1_dev->dev,
			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
			composed_frame_seq_no, dequeue_frame_seq_no);
		return;
	}
	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
		(dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
	dev_dbg(p1_dev->dev,
		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
		composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
}

void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
				   unsigned int frame_seq_no)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
			req->frame_params.frame_seq_no, frame_seq_no);

		/* Match by the en-queued request number */
		if (req->frame_params.frame_seq_no == frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
			list_del(&req->list);
			break;
		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space for frame drop */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
			dev_warn(cam->dev, "frame_seq:%d drop\n",
				 req->frame_params.frame_seq_no);
			list_del(&req->list);
		} else {
			break;
		}
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);

static irqreturn_t isp_irq_cam(int irq, void *data)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
	struct device *dev = p1_dev->dev;
	unsigned int dequeue_frame_seq_no;
	unsigned int irq_status, err_status, dma_status;
	unsigned long flags;

	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
	err_status = irq_status & INT_ST_MASK_CAM_ERR;
	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
	dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);

	/*
	 * In normal case, the next SOF ISR should come after HW PASS1 DONE
ISR.
	 * If these two ISRs come together, print warning msg to hint.
	 */
	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

	/* De-queue frame */
	if (irq_status & SW_PASS1_DON_ST) {
		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
					      dequeue_frame_seq_no);
		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
	}

	/* Save frame info. & update CQ address for frame HW en-queue */
	if (irq_status & SOF_INT_ST)
		isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);

	/* Check ISP error status */
	if (err_status) {
		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
		/* Show DMA errors in detail */
		if (err_status & DMA_ERR_ST)
			isp_irq_handle_dma_err(p1_dev);
	}

	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
		p1_dev->sof_count, irq_status, dma_status,
		dequeue_frame_seq_no);

	return IRQ_HANDLED;
}

> >
> > Below is our frame request handling in current design.
> >
> > 1. Buffer preparation
> > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > other meta histogram buffers (LCSO/LMVO) into one request.
> > - Accumulated one unique frame sequence number to each request and send
> > this request to the SCP composer to compose CQ (Command queue) buffer
> > via SCP_ISP_FRAME IPI command.
> > - CQ buffer is frame registers set. If ISP registers should be updated
> > per frame, these registers are configured in the CQ buffer, such as
> > frame sequence number, DMA addresses and tuning ISP registers.
> > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > its corresponding frame sequence number. Based on this, kernel driver
> > knows which request is ready to be en-queued and save this with
> > p1_dev->isp_ctx.composed_frame_id.
> 
> Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> already have a linked lists of requests that are composed and ready to
> be enqueued? Also, the request itself would contain its frame ID
> inside the driver request struct, right?
> 

Below is current implementation for frame request en-queued.
Before en-queued into HW by CQ, the request should be composed by SCP
composer.

a. mtk_cam_dev_req_try_queue()
- Insert the request into p1_dev->running_job_list
b. mtk_isp_req_enqueue()
- Assign new next frame ID to this request.
- Sending to SCP by workqueue
- This request is ready to compose
c. isp_tx_frame_worker()
- Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
- SCP composer will compose the buffer CQ for this request frame based
on struct mtk_p1_frame_param which includes frame ID.
- If scp_ipi_send() is returned, it means the request is composed done.
Or
d. isp_composer_handler()
- If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
- The request is composed done here.
e. isp_irq_handle_sof()
- In SOF timing, we will check there is any available composed CQ
buffers by comparing composed & current de-queued frame ID.

For p1_dev->running_job_list, we can't guarantee the requests are
composed until the end of step c. For step e, we need to know how many
available composed requests are ready to en-queued.

Do you suggest we add another new link-list to save these requests in
step c or we could update p1_dev->composed_frame_seq_no in step c and
remove the implementation in step d[1]?

[1]. isp_composer_handler() is mandatory callback function for SCP
sending API with sync mode design.

static void isp_composer_handler(void *data, unsigned int len, void
*priv)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
	struct mtk_isp_scp_p1_cmd *ipi_msg;

	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

	if (ipi_msg->cmd_id != ISP_CMD_ACK)
		return;

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}
}

> > - The maximum number of CQ buffers in SCP is 3.
> >
> > 2. Buffer en-queue flow
> > - In order to configure correct CQ buffer setting before next SQF event,
> > it is depended on by MTK ISP P1 HW CQ mechanism.
> > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > when HW_PASS1_DON_ST is received which means DMA output is done.
> > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > bypass.
> > - So we will check available CQ buffers by comparing composed frame
> > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > - If there are available CQ buffers, update the CQ base address to the
> > next CQ buffer address based on current de-enqueue frame sequence
> > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > request until request is done.
> > - But, for the first request, it is loaded into HW manually when
> > streaming is on for better performance.
> >
> > 3. Buffer de-queue flow
> > - We will use frame sequence number to decide which request is ready to
> > de-queue.
> > - We will save some important register setting from ISP HW when SOF is
> > received. This is because the ISP HW starts to output the data with the
> > corresponding settings, especially frame sequence number setting.
> 
> Could you explain a bit more about these important register settings?
> When does the hardware update the values in the register to new ones?
> At SOF?
> 

Sorry about my words.
In the current implementation, we just save frame ID.


> > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > done. So we could call isp_deque_request_frame with frame sequence
> > number to de-queue frame to VB2
> 
> What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> 

This is explained above.

> > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > AAO/AFO non-request buffers are en-queued without request buffers at the
> > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> 
> Do we have a case like this? Wouldn't we normally always want to
> bundle AAO/AFO buffers with frame buffers?
> 

For upstream driver, we will remove non-request design.

> > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > we may try to send another request to SCP for composing.
> 
> Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> buffers, right?
> 

We expected the the life cycle of CQ buffer is same as frame request.
So SW_PASS1_DON_ST is good timing to re-queue the next request to
compose.
For the CQ operations, we will explain later.

> >
> > Hopefully, my explanation is helpful for better understanding our
> > implementation. If you still have any questions, please let me know.
> >
> 
> Yes, it's more clear now, thanks. Still some more comments above, though.
> 
> > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +   } else {
> > > > +           if (irq_status & SOF_INT_ST) {
> > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +           }
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > +   }
> > >
> > > The if and else blocks do almost the same things just in different order. Is
> > > it really expected?
> > >
> >
> > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > any de-queued frame and update the next frame setting later.
> > Normally, this is a corner case or system performance issue.
> 
> So it sounds like HW_PASS1_DON_ST means that all data from current
> frame has been written, right? If I understand your explanation above
> correctly, that would mean following handling of each interrupt:
> 
> HW_PASS1_DON_ST:
>  - CQ executes with next CQ buffer to prepare for next frame. <- how
> is this handled? does the CQ hardware automatically receive this event
> from the ISP hadware?
>  - return VB2 buffers,
>  - complete requests.
> 
> SOF_INT_ST:
>  - send VSYNC event to userspace,
>  - program next CQ buffer to CQ,
> 
> SW_PASS1_DON_ST:
>  - reclaim CQ buffer and enqueue next frame to composing if available
> 

Sorry for our implementation of HW_PASS1_DON_ST.
It is confusing. 
Below is the revised version based on your conclusion.
So in our new implemmenation, we just handle SOF_INT_ST &
SW_PASS1_DON_ST events. We just add one warning message for
HW_PASS1_DON_ST
 
HW_PASS1_DON_ST:
- CQ executes with next CQ buffer to prepare for next frame.
 
SOF_INT_ST:
- send VSYNC event to userspace,
- program next CQ buffer to CQ,
 
SW_PASS1_DON_ST:
- reclaim CQ buffer and enqueue next frame to composing if available
- return VB2 buffers,
- complete requests.

For CQ HW operations, it is listed below:

a. The CQ buffer has two kinds of information
 - Which ISP registers needs to be updated.
 - Where the corresponding ISP register data to be read.
b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
event periodically. 
 - Normally, if the ISP HW receives the completed frame and it will
trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
-  So the CQ buffer loading is performed by ISP HW automatically.
c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
to decide which CQ buffer is loaded.
   - So we configure the next CQ base address in SOF.
d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
and update the ISP register values into HW.
   - SCP composer will compose one dummy CQ buffer and assign it to
REG_CQ_THR0_BASEADDR of each CQ buffer.
   - Dummy CQ buffer has no updated ISP registers comparing with other
CQ buffers.
   - With this design, if there is no updated new CQ buffer by driver
which may be caused no en-queue frames from user space. The CQ HW will
load dummy CQ buffer and do nothing.
f. The CQ buffer loading is guaranteed by HW to finish before the next
SOF.

> >
> > Btw, we will revise the above implementation as below.
> >
> >
> > if (irq_status & SOF_INT_ST)
> >         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> >                                              dequeue_frame_seq_no);
> >
> > /* Sof, done order check */
> > if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> >         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> >
> > /* Notify IRQ event and de-enqueue frame */
> > irq_handle_notify_event(p1_dev, irq_status, dma_status);
> 
> Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?
> 
> [snip]

Yes, in the new version, we will add SW_PASS1_DON_ST check before
calling mtk_cam_dev_dequeue_req_frame function.

> > > > +/* ISP P1 interface functions */
> > > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   int ret;
> > > > +
> > > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > > > +
> > > > +   ret = isp_init_context(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > >
> > > The above function doesn't really seem to be related to power management.
> > > Should it be called from subdev stream on?
> > >
> >
> > We will rename this function to mtk_isp_hw_init.
> > But, it will be called when the first video node is streamed on.
> > This is because we need to initialize the HW firstly for sub-device
> > stream-on performance.  We need to send some IPI commands, such as
> > ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> > timing.
> 
> What performance do you mean here? The time between first video node
> stream on and last video node stream on should be really short. Are
> you seeing some long delays there?
> 
> That said, doing it when the first video node starts streaming is okay.
> 
> [snip]

Ok, for IPI command sending performance issue is gone with removing the
design of non-request mode of 3A buffers. We could skip this question.

> > > > +   /* Use pure RAW as default HW path */
> > > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > > +
> > > > +   isp_composer_hw_init(dev);
> > > > +   /* Check enabled DMAs which is configured by media setup */
> > > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> > >
> > > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > > necessary?
> > >
> >
> > Yes, it is necessary for non-request buffers design for Camera 3A video
> > nodes. For 3A video nodes, we just want to know which 3A video nodes are
> > enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> > format from user space. So we just pass the enabled DMA information from
> > kernel to SCP only. With 3A enabled DMA information, we could configure
> > related 3A registers in SCP.
> 
> We should try to remove this non-request mode. Let's continue
> discussion on the other patch where I brought this topic.
> 
> [snip]

Ok. We will remove the non-request in next patch.

> > > > +int mtk_isp_power_release(struct device *dev)
> > > > +{
> > > > +   isp_composer_hw_deinit(dev);
> > > > +   isp_uninit_context(dev);
> > >
> > > These two don't seem to be related to power either.
> > >
> > > Instead, I don't see anything that could undo the rproc_boot() operation
> > > here.
> > >
> >
> > We will rename this function to mtk_isp_hw_release.
> > We will also add rproc_shutdown function call here.
> >
> > int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > {
> >         struct device *dev = cam->dev;
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         isp_composer_hw_deinit(p1_dev);
> >         pm_runtime_put_sync_autosuspend(dev);
> 
> Note that for autosuspend to work correctly, you also need to call
> pm_runtime_mark_last_busy() before this one.
> 
> [snip]

Ok, thanks for your hint.

> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   struct p1_frame_param frameparams;
> > > > +   struct mtk_isp_queue_job *framejob;
> > > > +   struct media_request_object *obj, *obj_safe;
> > > > +   struct vb2_buffer *vb;
> > > > +   struct mtk_cam_dev_buffer *buf;
> > > > +
> > > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > >
> > > This allocation shouldn't be needed. The structure should be already a part
> > > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > > even say that the contents of this struct should be just moved to that one
> > > to avoid overabstracting.
> > >
> >
> > For this function, we will apply the new design from P2 driver's
> > comment. Here is the new implementation.
> >
> > struct mtk_cam_dev_request {
> >         struct media_request req;
> >         struct mtk_p1_frame_param frame_params;
> >         struct work_struct frame_work;
> >         struct list_head list;
> >         atomic_t buf_count;
> > };
> >
> > void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >                          struct mtk_cam_dev_request *req)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >         int ret;
> >
> >         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
> >         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
> >         if (!ret)
> >                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
> >                         req->frame_params.frame_seq_no, ret);
> >         else
> >                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >                         req->req.debug_str, req->frame_params.frame_seq_no,
> >                         atomic_read(&cam->running_job_count));
> 
> It shouldn't be possible for queue_work() to fail here. We just
> received a brand new request from the Request API core and called
> INIT_WORK() on the work struct.
> 
> [snip]

Ok, got it. We will remove this unnecessary error checking.

> > > > +   enable_sys_clock(p1_dev);
> > > > +
> > > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > > +   if (p1_dev->cam_dev.streaming) {
> > > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > > +           /* Enable CMOS */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           writel((reg_val | CMOS_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           /* Enable VF */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > > +           writel((reg_val | VFDATA_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > > +   }
> > >
> > > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > > Would the code above continue all the capture from the next buffer, as
> > > queued by the userspace before the suspend?
> > >
> >
> > Yes, we will test it.
> > 1. SCP buffers are kept by SCP processor
> > 2. ISP registers are still kept even if ISP clock is disable.
> >
> 
> That said, during system suspend, it would be more than just ISP clock
> disabled. I'd expect that the ISP power domain would be powered off.
> However, if we ensure that the ISP completes before suspend, I guess
> that after the resume the next frame CQ buffer would reprogram all the
> registers, right?
> 
> Also, would SCP always keep running in system suspend?
> 
> [snip]

Q1. Firstly, we will make sure the ISP complete the current frame before
suspend. For the resume behavior, we will check internally.

Q2. Sorry for my wrong reply. The CQ buffer data should be kept in
memory, not SCP. SCP should also be stopped when system is suspend.

> > > > +
> > > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > >
> > > I think we want to start from 0 here?
> > >
> >
> > Because of single CAM support, we will revise our DTS tree to support
> > single CAM only.
> 
> Note that DT bindings should describe the hardware not the driver. So
> please design the bindings in a way that would cover all the cameras,
> even if the driver only takes the information needed to handle 1.
> 

Ok, we will cover all Camera ISP P1 HW units in the DT.

> > So we could remove this loop and check the CAM-B HW
> > information here. Here is below new function.
> >
> > static int mtk_isp_probe(struct platform_device *pdev)
> > {
> >         /* List of clocks required by isp cam */
> >         static const char * const clk_names[] = {
> >                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> >         };
> >         struct mtk_isp_p1_device *p1_dev;
> >         struct device *dev = &pdev->dev;
> >         struct resource *res;
> >         int irq, ret, i;
> >
> >         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> >         if (!p1_dev)
> >                 return -ENOMEM;
> >
> >         p1_dev->dev = dev;
> >         dev_set_drvdata(dev, p1_dev);
> >
> >         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >         p1_dev->regs = devm_ioremap_resource(dev, res);
> >         if (IS_ERR(p1_dev->regs)) {
> >                 dev_err(dev, "Failed platform resources map\n");
> >                 return PTR_ERR(p1_dev->regs);
> >         }
> >         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> >
> >         irq = platform_get_irq(pdev, 0);
> >         if (!irq) {
> >                 dev_err(dev, "Missing IRQ resources data\n");
> >                 return -ENODEV;
> >         }
> >         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
> >                                dev_name(dev), p1_dev);
> >         if (ret) {
> >                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
> >                         dev->of_node->name, irq);
> >                 return ret;
> >         }
> >         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
> >         spin_lock_init(&p1_dev->spinlock_irq);
> >
> >         p1_dev->num_clks = ARRAY_SIZE(clk_names);
> >         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> >                                     sizeof(*p1_dev->clks), GFP_KERNEL);
> >         if (!p1_dev->clks)
> >                 return -ENOMEM;
> >
> >         for (i = 0; i < p1_dev->num_clks; ++i)
> >                 p1_dev->clks[i].id = clk_names[i];
> >
> >         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> >         if (ret) {
> >                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> >                 return ret;
> >         }
> >
> >         ret = isp_setup_scp_rproc(p1_dev, pdev);
> >         if (ret)
> >                 return ret;
> >
> >         pm_runtime_enable(dev);
> 
> We also need to call pm_runtime_use_autosuspend() and
> pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
> suggest an autosuspend delay equal to around 2x the time that's needed
> to stop and start streaming in total.
> 
> [snip]

O, we will add these function calls.

> > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > >
> > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > different. Runtime PM ops would be called when the hardware is idle already
> > > or is about to become active. System PM ops would be called at system power
> > > state change and the hardware might be both idle or active. Please also see
> > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > >
> >
> > Here is the new implementation. It should be clear to show the
> > difference between system and runtime PM ops.
> >
> > static const struct dev_pm_ops mtk_isp_pm_ops = {
> >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> >                                 pm_runtime_force_resume)
> >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > NULL)
> > };
> 
> That's still not correct. In runtime suspend/resume ops we already are
> not streaming anymore, because we call pm_runtime_get/put_*() when
> starting and stopping streaming. In system suspend/resume ops we might
> be streaming and that's when we need to stop the hardware and wait for
> it to finish. Please implement these ops separately.
> 
> Best regards,
> Tomasz


Ok, got your point.
Below is the new implementation for your review.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Check ISP is streaming or not */
	if (!p1_dev->cam_dev.streaming)
		goto done;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "Cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

done:
	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Force ISP HW to resume if needed */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	if (!p1_dev->cam_dev.streaming)
		return 0;

	/* Enable CMOS */
	dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "enable clock\n");
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
			   NULL)
};

Best regards,


Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-07-26  7:23             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  7:23 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> .Hi Jungo,
> 
> On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > > > This patch adds the Mediatek ISP P1 HW control device driver.
> > > > It handles the ISP HW configuration, provides interrupt handling and
> > > > initializes the V4L2 device nodes and other functions.
> > > >
> > > > (The current metadata interface used in meta input and partial
> > > > meta nodes is only a temporary solution to kick off the driver
> > > > development and is not ready to be reviewed yet.)
> > > >
> > > > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > > > ---
> > > >  .../platform/mtk-isp/isp_50/cam/Makefile      |    1 +
> > > >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |  126 ++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 1087 +++++++++++++++++
> > > >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  243 ++++
> > > >  4 files changed, 1457 insertions(+)
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > > >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > > >
> > >
> > > Thanks for the patch! Please see my comments inline.
> > >
> > > [snip]
> > >
> >
> > Thanks for your comments. Please check my replies inline.
> >
> 
> Thanks! I'll snip anything I don't have further comments on.
> 
> [snip]

Ok, got it.

> > > > +/* META */
> > > > +#define REG_META0_VB2_INDEX                0x14dc
> > > > +#define REG_META1_VB2_INDEX                0x151c
> > >
> > > I don't believe these registers are really for VB2 indexes.
> > >
> >
> > MTK P1 ISP HW supports frame header spare registers for each DMA, such
> > as CAM_DMA_FH_AAO_SPARE or CAM_DMA_FH_AFO_SPARE. We could save some
> > frame information in these ISP registers. In this case, we save META0
> > VB2 index into AAO FH spare register and META1 VB2 index into AFO FH
> > spare register. These implementation is designed for non-request 3A
> > DMAs. These VB2 indexes are sent in ISP_CMD_ENQUEUE_META command of
> > mtk_isp_enqueue function. So we just call CAM_DMA_FH_AAO_SPARE as
> > REG_META0_VB2_INDEX for easy understanding.
> 
> Unfortunately it's not a good idea to mix hardware concepts with
> naming specific to the OS the driver is written for. Better to keep
> the hardware naming, e.g. CAM_DMA_FH_AAO_SPARE.
> 

Ok, got your point. We will pay attention in next time.
Moreover, we will remove AAO/AFO non-request design in next patch.
So these codes will also be removed.

> > Moreover, if we only need to
> > support request mode, we should remove this here.
> >
> > cmd_params.cmd_id = ISP_CMD_ENQUEUE_META;
> > cmd_params.meta_frame.enabled_dma = dma_port;
> > cmd_params.meta_frame.vb_index = buffer->vbb.vb2_buf.index;
> > cmd_params.meta_frame.meta_addr.iova = buffer->daddr;
> > cmd_params.meta_frame.meta_addr.scp_addr = buffer->scp_addr;
> >
> 
> Okay, removing sounds good to me. Let's keep the code simple.
> 
> [snip]

Thanks.

> > > > +
> > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > +
> > > > +   /* Sof, done order check */
> > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > +
> > > > +           /* Notify IRQ event and enqueue frame */
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > +           isp_dev->current_frame = hw_frame_num;
> > >
> > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > event?
> > >
> >
> > This is a another spare register for frame sequence number usage.
> > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > Is it a better understanding?
> 
> I'm sorry, unfortunately it's still not clear to me. Is it the
> sequence number of the frame that was just processed and returned to
> the kernel or the next frame that is going to be processed from now
> on?
> 

It is the next frame that is going to be proceed. 
We simplify the implementation of isp_irq_cam function. The hw_frame_num
is renamed to dequeue_frame_seq_no and saved this value from HW at
SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
next frame to be processed. If there is SW_PASS1_DON_ST, it means this
frame is processed done. We use this value to de-queue the frame request
and return buffers to VB2.

The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
HW_PASS1_DON_ST.

a. SW_PASS_DON_ST is designed for DMAs done event.
If there is no available DMA buffers en-queued into HW, there is no
SW_PADD_DON_ST.

b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
It is paired with SOF IRQ event, even if there is no available DMA
buffers.

static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
			       unsigned int dequeue_frame_seq_no)
{
	dma_addr_t base_addr = p1_dev->composer_iova;
	int composed_frame_seq_no =
atomic_read(&p1_dev->composed_frame_seq_no);
	unsigned int addr_offset;

	/* Send V4L2_EVENT_FRAME_SYNC event */
	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
	
	p1_dev->sof_count += 1;
	/* Save dequeue frame information */
	p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;

	/* Update CQ base address if needed */
	if (composed_frame_seq_no <= dequeue_frame_seq_no) {
		dev_dbg(p1_dev->dev,
			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
			composed_frame_seq_no, dequeue_frame_seq_no);
		return;
	}
	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
		(dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
	dev_dbg(p1_dev->dev,
		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
		composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
}

void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
				   unsigned int frame_seq_no)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
			req->frame_params.frame_seq_no, frame_seq_no);

		/* Match by the en-queued request number */
		if (req->frame_params.frame_seq_no == frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
			list_del(&req->list);
			break;
		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
			atomic_dec(&cam->running_job_count);
			/* Pass to user space for frame drop */
			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
			dev_warn(cam->dev, "frame_seq:%d drop\n",
				 req->frame_params.frame_seq_no);
			list_del(&req->list);
		} else {
			break;
		}
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);

static irqreturn_t isp_irq_cam(int irq, void *data)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
	struct device *dev = p1_dev->dev;
	unsigned int dequeue_frame_seq_no;
	unsigned int irq_status, err_status, dma_status;
	unsigned long flags;

	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
	err_status = irq_status & INT_ST_MASK_CAM_ERR;
	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
	dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);

	/*
	 * In normal case, the next SOF ISR should come after HW PASS1 DONE
ISR.
	 * If these two ISRs come together, print warning msg to hint.
	 */
	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);

	/* De-queue frame */
	if (irq_status & SW_PASS1_DON_ST) {
		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
					      dequeue_frame_seq_no);
		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
	}

	/* Save frame info. & update CQ address for frame HW en-queue */
	if (irq_status & SOF_INT_ST)
		isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);

	/* Check ISP error status */
	if (err_status) {
		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
		/* Show DMA errors in detail */
		if (err_status & DMA_ERR_ST)
			isp_irq_handle_dma_err(p1_dev);
	}

	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
		p1_dev->sof_count, irq_status, dma_status,
		dequeue_frame_seq_no);

	return IRQ_HANDLED;
}

> >
> > Below is our frame request handling in current design.
> >
> > 1. Buffer preparation
> > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > other meta histogram buffers (LCSO/LMVO) into one request.
> > - Accumulated one unique frame sequence number to each request and send
> > this request to the SCP composer to compose CQ (Command queue) buffer
> > via SCP_ISP_FRAME IPI command.
> > - CQ buffer is frame registers set. If ISP registers should be updated
> > per frame, these registers are configured in the CQ buffer, such as
> > frame sequence number, DMA addresses and tuning ISP registers.
> > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > its corresponding frame sequence number. Based on this, kernel driver
> > knows which request is ready to be en-queued and save this with
> > p1_dev->isp_ctx.composed_frame_id.
> 
> Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> already have a linked lists of requests that are composed and ready to
> be enqueued? Also, the request itself would contain its frame ID
> inside the driver request struct, right?
> 

Below is current implementation for frame request en-queued.
Before en-queued into HW by CQ, the request should be composed by SCP
composer.

a. mtk_cam_dev_req_try_queue()
- Insert the request into p1_dev->running_job_list
b. mtk_isp_req_enqueue()
- Assign new next frame ID to this request.
- Sending to SCP by workqueue
- This request is ready to compose
c. isp_tx_frame_worker()
- Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
- SCP composer will compose the buffer CQ for this request frame based
on struct mtk_p1_frame_param which includes frame ID.
- If scp_ipi_send() is returned, it means the request is composed done.
Or
d. isp_composer_handler()
- If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
- The request is composed done here.
e. isp_irq_handle_sof()
- In SOF timing, we will check there is any available composed CQ
buffers by comparing composed & current de-queued frame ID.

For p1_dev->running_job_list, we can't guarantee the requests are
composed until the end of step c. For step e, we need to know how many
available composed requests are ready to en-queued.

Do you suggest we add another new link-list to save these requests in
step c or we could update p1_dev->composed_frame_seq_no in step c and
remove the implementation in step d[1]?

[1]. isp_composer_handler() is mandatory callback function for SCP
sending API with sync mode design.

static void isp_composer_handler(void *data, unsigned int len, void
*priv)
{
	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
	struct mtk_isp_scp_p1_cmd *ipi_msg;

	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;

	if (ipi_msg->cmd_id != ISP_CMD_ACK)
		return;

	if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
		atomic_set(&p1_dev->composed_frame_seq_no,
			   ipi_msg->ack_info.frame_seq_no);
		dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
			p1_dev->composed_frame_seq_no);
	}
}

> > - The maximum number of CQ buffers in SCP is 3.
> >
> > 2. Buffer en-queue flow
> > - In order to configure correct CQ buffer setting before next SQF event,
> > it is depended on by MTK ISP P1 HW CQ mechanism.
> > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > when HW_PASS1_DON_ST is received which means DMA output is done.
> > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > bypass.
> > - So we will check available CQ buffers by comparing composed frame
> > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > - If there are available CQ buffers, update the CQ base address to the
> > next CQ buffer address based on current de-enqueue frame sequence
> > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > request until request is done.
> > - But, for the first request, it is loaded into HW manually when
> > streaming is on for better performance.
> >
> > 3. Buffer de-queue flow
> > - We will use frame sequence number to decide which request is ready to
> > de-queue.
> > - We will save some important register setting from ISP HW when SOF is
> > received. This is because the ISP HW starts to output the data with the
> > corresponding settings, especially frame sequence number setting.
> 
> Could you explain a bit more about these important register settings?
> When does the hardware update the values in the register to new ones?
> At SOF?
> 

Sorry about my words.
In the current implementation, we just save frame ID.


> > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > done. So we could call isp_deque_request_frame with frame sequence
> > number to de-queue frame to VB2
> 
> What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> 

This is explained above.

> > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > AAO/AFO non-request buffers are en-queued without request buffers at the
> > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> 
> Do we have a case like this? Wouldn't we normally always want to
> bundle AAO/AFO buffers with frame buffers?
> 

For upstream driver, we will remove non-request design.

> > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > we may try to send another request to SCP for composing.
> 
> Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> buffers, right?
> 

We expected the the life cycle of CQ buffer is same as frame request.
So SW_PASS1_DON_ST is good timing to re-queue the next request to
compose.
For the CQ operations, we will explain later.

> >
> > Hopefully, my explanation is helpful for better understanding our
> > implementation. If you still have any questions, please let me know.
> >
> 
> Yes, it's more clear now, thanks. Still some more comments above, though.
> 
> > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +   } else {
> > > > +           if (irq_status & SOF_INT_ST) {
> > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > +           }
> > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > +   }
> > >
> > > The if and else blocks do almost the same things just in different order. Is
> > > it really expected?
> > >
> >
> > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > any de-queued frame and update the next frame setting later.
> > Normally, this is a corner case or system performance issue.
> 
> So it sounds like HW_PASS1_DON_ST means that all data from current
> frame has been written, right? If I understand your explanation above
> correctly, that would mean following handling of each interrupt:
> 
> HW_PASS1_DON_ST:
>  - CQ executes with next CQ buffer to prepare for next frame. <- how
> is this handled? does the CQ hardware automatically receive this event
> from the ISP hadware?
>  - return VB2 buffers,
>  - complete requests.
> 
> SOF_INT_ST:
>  - send VSYNC event to userspace,
>  - program next CQ buffer to CQ,
> 
> SW_PASS1_DON_ST:
>  - reclaim CQ buffer and enqueue next frame to composing if available
> 

Sorry for our implementation of HW_PASS1_DON_ST.
It is confusing. 
Below is the revised version based on your conclusion.
So in our new implemmenation, we just handle SOF_INT_ST &
SW_PASS1_DON_ST events. We just add one warning message for
HW_PASS1_DON_ST
 
HW_PASS1_DON_ST:
- CQ executes with next CQ buffer to prepare for next frame.
 
SOF_INT_ST:
- send VSYNC event to userspace,
- program next CQ buffer to CQ,
 
SW_PASS1_DON_ST:
- reclaim CQ buffer and enqueue next frame to composing if available
- return VB2 buffers,
- complete requests.

For CQ HW operations, it is listed below:

a. The CQ buffer has two kinds of information
 - Which ISP registers needs to be updated.
 - Where the corresponding ISP register data to be read.
b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
event periodically. 
 - Normally, if the ISP HW receives the completed frame and it will
trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
-  So the CQ buffer loading is performed by ISP HW automatically.
c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
to decide which CQ buffer is loaded.
   - So we configure the next CQ base address in SOF.
d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
and update the ISP register values into HW.
   - SCP composer will compose one dummy CQ buffer and assign it to
REG_CQ_THR0_BASEADDR of each CQ buffer.
   - Dummy CQ buffer has no updated ISP registers comparing with other
CQ buffers.
   - With this design, if there is no updated new CQ buffer by driver
which may be caused no en-queue frames from user space. The CQ HW will
load dummy CQ buffer and do nothing.
f. The CQ buffer loading is guaranteed by HW to finish before the next
SOF.

> >
> > Btw, we will revise the above implementation as below.
> >
> >
> > if (irq_status & SOF_INT_ST)
> >         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev,
> >                                              dequeue_frame_seq_no);
> >
> > /* Sof, done order check */
> > if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> >         dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> >
> > /* Notify IRQ event and de-enqueue frame */
> > irq_handle_notify_event(p1_dev, irq_status, dma_status);
> 
> Don't we still need to do this conditionally, only if we got HW_PASS1_DON_ST?
> 
> [snip]

Yes, in the new version, we will add SW_PASS1_DON_ST check before
calling mtk_cam_dev_dequeue_req_frame function.

> > > > +/* ISP P1 interface functions */
> > > > +int mtk_isp_power_init(struct mtk_cam_dev *cam_dev)
> > > > +{
> > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   int ret;
> > > > +
> > > > +   ret = isp_setup_scp_rproc(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > > > +
> > > > +   ret = isp_init_context(p1_dev);
> > > > +   if (ret)
> > > > +           return ret;
> > >
> > > The above function doesn't really seem to be related to power management.
> > > Should it be called from subdev stream on?
> > >
> >
> > We will rename this function to mtk_isp_hw_init.
> > But, it will be called when the first video node is streamed on.
> > This is because we need to initialize the HW firstly for sub-device
> > stream-on performance.  We need to send some IPI commands, such as
> > ISP_CMD_INIT & ISP_CMD_CONFIG_META & ISP_CMD_ENQUEUE_META in this
> > timing.
> 
> What performance do you mean here? The time between first video node
> stream on and last video node stream on should be really short. Are
> you seeing some long delays there?
> 
> That said, doing it when the first video node starts streaming is okay.
> 
> [snip]

Ok, for IPI command sending performance issue is gone with removing the
design of non-request mode of 3A buffers. We could skip this question.

> > > > +   /* Use pure RAW as default HW path */
> > > > +   isp_ctx->isp_raw_path = ISP_PURE_RAW_PATH;
> > > > +   atomic_set(&p1_dev->cam_dev.streamed_node_count, 0);
> > > > +
> > > > +   isp_composer_hw_init(dev);
> > > > +   /* Check enabled DMAs which is configured by media setup */
> > > > +   isp_composer_meta_config(dev, get_enabled_dma_ports(cam_dev));
> > >
> > > Hmm, this seems to be also configured by isp_compoer_hw_config(). Are both
> > > necessary?
> > >
> >
> > Yes, it is necessary for non-request buffers design for Camera 3A video
> > nodes. For 3A video nodes, we just want to know which 3A video nodes are
> > enabled in ISP_CMD_CONFIG_META. In this stage, we may not know the image
> > format from user space. So we just pass the enabled DMA information from
> > kernel to SCP only. With 3A enabled DMA information, we could configure
> > related 3A registers in SCP.
> 
> We should try to remove this non-request mode. Let's continue
> discussion on the other patch where I brought this topic.
> 
> [snip]

Ok. We will remove the non-request in next patch.

> > > > +int mtk_isp_power_release(struct device *dev)
> > > > +{
> > > > +   isp_composer_hw_deinit(dev);
> > > > +   isp_uninit_context(dev);
> > >
> > > These two don't seem to be related to power either.
> > >
> > > Instead, I don't see anything that could undo the rproc_boot() operation
> > > here.
> > >
> >
> > We will rename this function to mtk_isp_hw_release.
> > We will also add rproc_shutdown function call here.
> >
> > int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > {
> >         struct device *dev = cam->dev;
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         isp_composer_hw_deinit(p1_dev);
> >         pm_runtime_put_sync_autosuspend(dev);
> 
> Note that for autosuspend to work correctly, you also need to call
> pm_runtime_mark_last_busy() before this one.
> 
> [snip]

Ok, thanks for your hint.

> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +   struct p1_frame_param frameparams;
> > > > +   struct mtk_isp_queue_job *framejob;
> > > > +   struct media_request_object *obj, *obj_safe;
> > > > +   struct vb2_buffer *vb;
> > > > +   struct mtk_cam_dev_buffer *buf;
> > > > +
> > > > +   framejob = kzalloc(sizeof(*framejob), GFP_ATOMIC);
> > >
> > > This allocation shouldn't be needed. The structure should be already a part
> > > of the mtk_cam_dev_request struct that wraps media_request. Actually. I'd
> > > even say that the contents of this struct should be just moved to that one
> > > to avoid overabstracting.
> > >
> >
> > For this function, we will apply the new design from P2 driver's
> > comment. Here is the new implementation.
> >
> > struct mtk_cam_dev_request {
> >         struct media_request req;
> >         struct mtk_p1_frame_param frame_params;
> >         struct work_struct frame_work;
> >         struct list_head list;
> >         atomic_t buf_count;
> > };
> >
> > void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >                          struct mtk_cam_dev_request *req)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >         int ret;
> >
> >         req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;
> >         INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >         ret = queue_work(p1_dev->composer_wq, &req->frame_work);
> >         if (!ret)
> >                 dev_dbg(cam->dev, "frame_no:%d queue_work failed\n",
> >                         req->frame_params.frame_seq_no, ret);
> >         else
> >                 dev_dbg(cam->dev, "Enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >                         req->req.debug_str, req->frame_params.frame_seq_no,
> >                         atomic_read(&cam->running_job_count));
> 
> It shouldn't be possible for queue_work() to fail here. We just
> received a brand new request from the Request API core and called
> INIT_WORK() on the work struct.
> 
> [snip]

Ok, got it. We will remove this unnecessary error checking.

> > > > +   enable_sys_clock(p1_dev);
> > > > +
> > > > +   /* V4L2 stream-on phase & restore HW stream-on status */
> > > > +   if (p1_dev->cam_dev.streaming) {
> > > > +           dev_dbg(dev, "Cam:%d resume,enable VF\n", module);
> > > > +           /* Enable CMOS */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           writel((reg_val | CMOS_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_SEN_MODE);
> > > > +           /* Enable VF */
> > > > +           reg_val = readl(isp_dev->regs + REG_TG_VF_CON);
> > > > +           writel((reg_val | VFDATA_EN_BIT),
> > > > +                  isp_dev->regs + REG_TG_VF_CON);
> > > > +   }
> > >
> > > Does the hardware keep all the state, e.g. queued buffers, during suspend?
> > > Would the code above continue all the capture from the next buffer, as
> > > queued by the userspace before the suspend?
> > >
> >
> > Yes, we will test it.
> > 1. SCP buffers are kept by SCP processor
> > 2. ISP registers are still kept even if ISP clock is disable.
> >
> 
> That said, during system suspend, it would be more than just ISP clock
> disabled. I'd expect that the ISP power domain would be powered off.
> However, if we ensure that the ISP completes before suspend, I guess
> that after the resume the next frame CQ buffer would reprogram all the
> registers, right?
> 
> Also, would SCP always keep running in system suspend?
> 
> [snip]

Q1. Firstly, we will make sure the ISP complete the current frame before
suspend. For the resume behavior, we will check internally.

Q2. Sorry for my wrong reply. The CQ buffer data should be kept in
memory, not SCP. SCP should also be stopped when system is suspend.

> > > > +
> > > > +   for (i = ISP_CAMSYS_CONFIG_IDX; i < ISP_DEV_NODE_NUM; i++) {
> > >
> > > I think we want to start from 0 here?
> > >
> >
> > Because of single CAM support, we will revise our DTS tree to support
> > single CAM only.
> 
> Note that DT bindings should describe the hardware not the driver. So
> please design the bindings in a way that would cover all the cameras,
> even if the driver only takes the information needed to handle 1.
> 

Ok, we will cover all Camera ISP P1 HW units in the DT.

> > So we could remove this loop and check the CAM-B HW
> > information here. Here is below new function.
> >
> > static int mtk_isp_probe(struct platform_device *pdev)
> > {
> >         /* List of clocks required by isp cam */
> >         static const char * const clk_names[] = {
> >                 "camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> >         };
> >         struct mtk_isp_p1_device *p1_dev;
> >         struct device *dev = &pdev->dev;
> >         struct resource *res;
> >         int irq, ret, i;
> >
> >         p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> >         if (!p1_dev)
> >                 return -ENOMEM;
> >
> >         p1_dev->dev = dev;
> >         dev_set_drvdata(dev, p1_dev);
> >
> >         res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >         p1_dev->regs = devm_ioremap_resource(dev, res);
> >         if (IS_ERR(p1_dev->regs)) {
> >                 dev_err(dev, "Failed platform resources map\n");
> >                 return PTR_ERR(p1_dev->regs);
> >         }
> >         dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> >
> >         irq = platform_get_irq(pdev, 0);
> >         if (!irq) {
> >                 dev_err(dev, "Missing IRQ resources data\n");
> >                 return -ENODEV;
> >         }
> >         ret = devm_request_irq(dev, irq, isp_irq_cam, IRQF_SHARED,
> >                                dev_name(dev), p1_dev);
> >         if (ret) {
> >                 dev_err(dev, "req_irq fail, dev:%s irq=%d\n",
> >                         dev->of_node->name, irq);
> >                 return ret;
> >         }
> >         dev_dbg(dev, "Reg. irq=%d, isr:%s\n", irq, dev_driver_string(dev));
> >         spin_lock_init(&p1_dev->spinlock_irq);
> >
> >         p1_dev->num_clks = ARRAY_SIZE(clk_names);
> >         p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> >                                     sizeof(*p1_dev->clks), GFP_KERNEL);
> >         if (!p1_dev->clks)
> >                 return -ENOMEM;
> >
> >         for (i = 0; i < p1_dev->num_clks; ++i)
> >                 p1_dev->clks[i].id = clk_names[i];
> >
> >         ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> >         if (ret) {
> >                 dev_err(dev, "cannot get isp cam clock:%d\n", ret);
> >                 return ret;
> >         }
> >
> >         ret = isp_setup_scp_rproc(p1_dev, pdev);
> >         if (ret)
> >                 return ret;
> >
> >         pm_runtime_enable(dev);
> 
> We also need to call pm_runtime_use_autosuspend() and
> pm_runtime_set_autosuspend_delay() before enabling runtime PM. I'd
> suggest an autosuspend delay equal to around 2x the time that's needed
> to stop and start streaming in total.
> 
> [snip]

O, we will add these function calls.

> > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > >
> > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > different. Runtime PM ops would be called when the hardware is idle already
> > > or is about to become active. System PM ops would be called at system power
> > > state change and the hardware might be both idle or active. Please also see
> > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > >
> >
> > Here is the new implementation. It should be clear to show the
> > difference between system and runtime PM ops.
> >
> > static const struct dev_pm_ops mtk_isp_pm_ops = {
> >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> >                                 pm_runtime_force_resume)
> >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > NULL)
> > };
> 
> That's still not correct. In runtime suspend/resume ops we already are
> not streaming anymore, because we call pm_runtime_get/put_*() when
> starting and stopping streaming. In system suspend/resume ops we might
> be streaming and that's when we need to stop the hardware and wait for
> it to finish. Please implement these ops separately.
> 
> Best regards,
> Tomasz


Ok, got your point.
Below is the new implementation for your review.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Check ISP is streaming or not */
	if (!p1_dev->cam_dev.streaming)
		goto done;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "Cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

done:
	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	/* Force ISP HW to resume if needed */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	if (!p1_dev->cam_dev.streaming)
		return 0;

	/* Enable CMOS */
	dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	dev_dbg(dev, "enable clock\n");
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "cannot enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

static const struct dev_pm_ops mtk_isp_pm_ops = {
	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
			   NULL)
};

Best regards,


Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-26  5:15                       ` Tomasz Figa
  (?)
@ 2019-07-26  7:41                           ` Christoph Hellwig
  -1 siblings, 0 replies; 388+ messages in thread
From: Christoph Hellwig @ 2019-07-26  7:41 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Frederic Chen (陳俊元),
	list-Y9sIeH5OGRo

On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> Could you try dma_get_sgtable() with the SCP struct device and then
> dma_map_sg() with the P1 struct device?

Please don't do that.  dma_get_sgtable is a pretty broken API (see
the common near the arm implementation) and we should not add more
users of it.  If you want a piece of memory that can be mapped to
multiple devices allocate it using alloc_pages and then just map
it to each device.

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  7:41                           ` Christoph Hellwig
  0 siblings, 0 replies; 388+ messages in thread
From: Christoph Hellwig @ 2019-07-26  7:41 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Jungo Lin, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> Could you try dma_get_sgtable() with the SCP struct device and then
> dma_map_sg() with the P1 struct device?

Please don't do that.  dma_get_sgtable is a pretty broken API (see
the common near the arm implementation) and we should not add more
users of it.  If you want a piece of memory that can be mapped to
multiple devices allocate it using alloc_pages and then just map
it to each device.

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  7:41                           ` Christoph Hellwig
  0 siblings, 0 replies; 388+ messages in thread
From: Christoph Hellwig @ 2019-07-26  7:41 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Matthias Brugger

On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> Could you try dma_get_sgtable() with the SCP struct device and then
> dma_map_sg() with the P1 struct device?

Please don't do that.  dma_get_sgtable is a pretty broken API (see
the common near the arm implementation) and we should not add more
users of it.  If you want a piece of memory that can be mapped to
multiple devices allocate it using alloc_pages and then just map
it to each device.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-26  7:41                           ` Christoph Hellwig
  (?)
@ 2019-07-26  7:42                             ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  7:42 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list

On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > Could you try dma_get_sgtable() with the SCP struct device and then
> > dma_map_sg() with the P1 struct device?
>
> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> the common near the arm implementation) and we should not add more
> users of it.  If you want a piece of memory that can be mapped to
> multiple devices allocate it using alloc_pages and then just map
> it to each device.

Thanks for taking a look at this thread.

Unfortunately that wouldn't work. We have a specific reserved memory
pool that is the only memory area accessible to one of the devices.
Any idea how to handle this?

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  7:42                             ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  7:42 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Jungo Lin, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > Could you try dma_get_sgtable() with the SCP struct device and then
> > dma_map_sg() with the P1 struct device?
>
> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> the common near the arm implementation) and we should not add more
> users of it.  If you want a piece of memory that can be mapped to
> multiple devices allocate it using alloc_pages and then just map
> it to each device.

Thanks for taking a look at this thread.

Unfortunately that wouldn't work. We have a specific reserved memory
pool that is the only memory area accessible to one of the devices.
Any idea how to handle this?

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26  7:42                             ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26  7:42 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Matthias Brugger

On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > Could you try dma_get_sgtable() with the SCP struct device and then
> > dma_map_sg() with the P1 struct device?
>
> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> the common near the arm implementation) and we should not add more
> users of it.  If you want a piece of memory that can be mapped to
> multiple devices allocate it using alloc_pages and then just map
> it to each device.

Thanks for taking a look at this thread.

Unfortunately that wouldn't work. We have a specific reserved memory
pool that is the only memory area accessible to one of the devices.
Any idea how to handle this?

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
  2019-07-25 10:56             ` [RFC,v3 " Tomasz Figa
  (?)
@ 2019-07-26  8:07               ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  8:07 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab, list

Hi, Tomasz:

On Thu, 2019-07-25 at 19:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> [snip]
> > > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > > +   }
> > > > +
> > > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > > +   }
> > > > +   mutex_unlock(&isp_ctx->lock);
> > > > +
> > > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> > >
> > > No need to use the sync variant.
> > >
> >
> > We don't get this point. If we will call pm_runtime_get_sync in
> > mtk_isp_hw_init function, will we need to call
> > pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> > As we know, we should call runtime pm functions in pair.
> >
> 
> My point is that pm_runtime_put_sync() is only needed if one wants the
> runtime count to be decremented after the function returns. Normally
> there is no need to do so and one would call pm_runtime_put(), or if
> autosuspend is used, pm_runtime_put_autosuspend() (note there is no
> "sync" in the name).
> 
> [snip]

Ok, got your point.
We will change to use pm_runtime_put_autosuspend() which has ASYNC flag.

> > > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > > +{
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > > +   struct device *dev = &p1_dev->pdev->dev;
> > > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > > +
> > > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > >
> > > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > > contain some extra bytes at the end.)
> > >
> >
> > The len parameter is the actual sending bytes from SCP to kernel.
> > In the runtime, it is only 6 bytes for isp_ack_info command
> > However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> > union structure.
> >
> 
> That said we still should check if len is enough to cover the data
> we're accessing below.
> 

Ok, we will add the len checking before accessing the data.

> > > > +
> > > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > > +           return;
> > > > +
> > > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > > +           dev_dbg(dev, "ack frame_num:%d",
> > > > +                   ipi_msg->ack_info.frame_seq_no);
> > > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > > +                      ipi_msg->ack_info.frame_seq_no);
> > >
> > > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > > frames and we should handle that somehow.
> > >
> >
> > No, we use isp_ctx->composed_frame_id to save which frame sequence
> > number are composed done in SCP. In new design, we will move this from
> > isp_ctx to p1_dev.
> 
> But we compose the frames in order, don't we? Wouldn't every composed
> frame would be just previous frame ID + 1?
> 
> [snip]

Yes, we compose the frames in order.
At the same time, we already increased "frame ID + 1" in
mtk_isp_req_enqueue() for each new request before sending to SCP for
composing. After receiving the ACK from SCP, we think the frame ID is
composed done and save by isp_ctx->composed_frame_id(v3).

[RFC v3]
void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
{
	...
	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;

[RFC v4]
void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

 

> > > > +void isp_composer_hw_init(struct device *dev)
> > > > +{
> > > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +
> > > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> > >
> > > Should we also specify the size of the buffer? Otherwise we could end up
> > > with some undetectable overruns.
> > >
> >
> > The size of SCP composer's memory is fixed to 0x200000.
> > Is it necessary to specify the size of this buffer?
> >
> > #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
> >
> > ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> >
> 
> Okay, but please add a comment saying that this is an implicit
> requirement of the firmware.
> 
> Best regards,
> Tomasz

Ok, we will add comments.

Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-26  8:07               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  8:07 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi, Tomasz:

On Thu, 2019-07-25 at 19:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> [snip]
> > > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > > +   }
> > > > +
> > > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > > +   }
> > > > +   mutex_unlock(&isp_ctx->lock);
> > > > +
> > > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> > >
> > > No need to use the sync variant.
> > >
> >
> > We don't get this point. If we will call pm_runtime_get_sync in
> > mtk_isp_hw_init function, will we need to call
> > pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> > As we know, we should call runtime pm functions in pair.
> >
> 
> My point is that pm_runtime_put_sync() is only needed if one wants the
> runtime count to be decremented after the function returns. Normally
> there is no need to do so and one would call pm_runtime_put(), or if
> autosuspend is used, pm_runtime_put_autosuspend() (note there is no
> "sync" in the name).
> 
> [snip]

Ok, got your point.
We will change to use pm_runtime_put_autosuspend() which has ASYNC flag.

> > > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > > +{
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > > +   struct device *dev = &p1_dev->pdev->dev;
> > > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > > +
> > > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > >
> > > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > > contain some extra bytes at the end.)
> > >
> >
> > The len parameter is the actual sending bytes from SCP to kernel.
> > In the runtime, it is only 6 bytes for isp_ack_info command
> > However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> > union structure.
> >
> 
> That said we still should check if len is enough to cover the data
> we're accessing below.
> 

Ok, we will add the len checking before accessing the data.

> > > > +
> > > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > > +           return;
> > > > +
> > > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > > +           dev_dbg(dev, "ack frame_num:%d",
> > > > +                   ipi_msg->ack_info.frame_seq_no);
> > > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > > +                      ipi_msg->ack_info.frame_seq_no);
> > >
> > > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > > frames and we should handle that somehow.
> > >
> >
> > No, we use isp_ctx->composed_frame_id to save which frame sequence
> > number are composed done in SCP. In new design, we will move this from
> > isp_ctx to p1_dev.
> 
> But we compose the frames in order, don't we? Wouldn't every composed
> frame would be just previous frame ID + 1?
> 
> [snip]

Yes, we compose the frames in order.
At the same time, we already increased "frame ID + 1" in
mtk_isp_req_enqueue() for each new request before sending to SCP for
composing. After receiving the ACK from SCP, we think the frame ID is
composed done and save by isp_ctx->composed_frame_id(v3).

[RFC v3]
void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
{
	...
	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;

[RFC v4]
void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

 

> > > > +void isp_composer_hw_init(struct device *dev)
> > > > +{
> > > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +
> > > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> > >
> > > Should we also specify the size of the buffer? Otherwise we could end up
> > > with some undetectable overruns.
> > >
> >
> > The size of SCP composer's memory is fixed to 0x200000.
> > Is it necessary to specify the size of this buffer?
> >
> > #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
> >
> > ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> >
> 
> Okay, but please add a comment saying that this is an implicit
> requirement of the firmware.
> 
> Best regards,
> Tomasz

Ok, we will add comments.

Best regards,


Jungo





^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication
@ 2019-07-26  8:07               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26  8:07 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Thu, 2019-07-25 at 19:56 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Sun, Jul 21, 2019 at 11:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> [snip]
> > > > +           wake_up_interruptible(&isp_ctx->composer_tx_thread.wq);
> > > > +           isp_ctx->composer_tx_thread.thread = NULL;
> > > > +   }
> > > > +
> > > > +   if (isp_ctx->composer_deinit_thread.thread) {
> > > > +           wake_up(&isp_ctx->composer_deinit_thread.wq);
> > > > +           isp_ctx->composer_deinit_thread.thread = NULL;
> > > > +   }
> > > > +   mutex_unlock(&isp_ctx->lock);
> > > > +
> > > > +   pm_runtime_put_sync(&p1_dev->pdev->dev);
> > >
> > > No need to use the sync variant.
> > >
> >
> > We don't get this point. If we will call pm_runtime_get_sync in
> > mtk_isp_hw_init function, will we need to call
> > pm_runtime_put_sync_autosuspend in mtk_isp_hw_release in next patch?
> > As we know, we should call runtime pm functions in pair.
> >
> 
> My point is that pm_runtime_put_sync() is only needed if one wants the
> runtime count to be decremented after the function returns. Normally
> there is no need to do so and one would call pm_runtime_put(), or if
> autosuspend is used, pm_runtime_put_autosuspend() (note there is no
> "sync" in the name).
> 
> [snip]

Ok, got your point.
We will change to use pm_runtime_put_autosuspend() which has ASYNC flag.

> > > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > > > +{
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = (struct mtk_isp_p1_ctx *)priv;
> > > > +   struct isp_p1_device *p1_dev = p1_ctx_to_dev(isp_ctx);
> > > > +   struct device *dev = &p1_dev->pdev->dev;
> > > > +   struct mtk_isp_scp_p1_cmd *ipi_msg;
> > > > +
> > > > +   ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > >
> > > Should we check that len == sizeof(*ipi_msg)? (Or at least >=, if data could
> > > contain some extra bytes at the end.)
> > >
> >
> > The len parameter is the actual sending bytes from SCP to kernel.
> > In the runtime, it is only 6 bytes for isp_ack_info command
> > However, sizeof(*ipi_msg) is large due to struct mtk_isp_scp_p1_cmd is
> > union structure.
> >
> 
> That said we still should check if len is enough to cover the data
> we're accessing below.
> 

Ok, we will add the len checking before accessing the data.

> > > > +
> > > > +   if (ipi_msg->cmd_id != ISP_CMD_ACK)
> > > > +           return;
> > > > +
> > > > +   if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
> > > > +           dev_dbg(dev, "ack frame_num:%d",
> > > > +                   ipi_msg->ack_info.frame_seq_no);
> > > > +           atomic_set(&isp_ctx->composed_frame_id,
> > > > +                      ipi_msg->ack_info.frame_seq_no);
> > >
> > > I suppose we are expecting here that ipi_msg->ack_info.frame_seq_no would be
> > > just isp_ctx->composed_frame_id + 1, right? If not, we probably dropped some
> > > frames and we should handle that somehow.
> > >
> >
> > No, we use isp_ctx->composed_frame_id to save which frame sequence
> > number are composed done in SCP. In new design, we will move this from
> > isp_ctx to p1_dev.
> 
> But we compose the frames in order, don't we? Wouldn't every composed
> frame would be just previous frame ID + 1?
> 
> [snip]

Yes, we compose the frames in order.
At the same time, we already increased "frame ID + 1" in
mtk_isp_req_enqueue() for each new request before sending to SCP for
composing. After receiving the ACK from SCP, we think the frame ID is
composed done and save by isp_ctx->composed_frame_id(v3).

[RFC v3]
void mtk_isp_req_enqueue(struct device *dev, struct media_request *req)
{
	...
	frameparams.frame_seq_no = isp_ctx->frame_seq_no++;

[RFC v4]
void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
			 struct mtk_cam_dev_request *req)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);

	/* Accumulated frame sequence number */
	req->frame_params.frame_seq_no = ++p1_dev->enqueue_frame_seq_no;

 

> > > > +void isp_composer_hw_init(struct device *dev)
> > > > +{
> > > > +   struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > > > +   struct isp_p1_device *p1_dev = get_p1_device(dev);
> > > > +   struct mtk_isp_p1_ctx *isp_ctx = &p1_dev->isp_ctx;
> > > > +
> > > > +   memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > > > +   composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > > > +   composer_tx_cmd.frameparam.hw_module = isp_ctx->isp_hw_module;
> > > > +   composer_tx_cmd.frameparam.cq_addr.iova = isp_ctx->scp_mem_iova;
> > > > +   composer_tx_cmd.frameparam.cq_addr.scp_addr = isp_ctx->scp_mem_pa;
> > >
> > > Should we also specify the size of the buffer? Otherwise we could end up
> > > with some undetectable overruns.
> > >
> >
> > The size of SCP composer's memory is fixed to 0x200000.
> > Is it necessary to specify the size of this buffer?
> >
> > #define MTK_ISP_COMPOSER_MEM_SIZE 0x200000
> >
> > ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> >                         MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> >
> 
> Okay, but please add a comment saying that this is an implicit
> requirement of the firmware.
> 
> Best regards,
> Tomasz

Ok, we will add comments.

Best regards,


Jungo





_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-26  7:42                             ` Tomasz Figa
  (?)
  (?)
@ 2019-07-26 11:04                                 ` Robin Murphy
  -1 siblings, 0 replies; 388+ messages in thread
From: Robin Murphy @ 2019-07-26 11:04 UTC (permalink / raw)
  To: Tomasz Figa, Christoph Hellwig
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Matthias Brugger, Frankie Chiu (邱文凱),
	list-Y9sIeH5OGRo@public.gmane.org:IOMMU DRIVERS, Hans Verkuil,
	Jungo Lin, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Mauro Carvalho Chehab

On 26/07/2019 08:42, Tomasz Figa wrote:
> On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
>>
>> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
>>> Could you try dma_get_sgtable() with the SCP struct device and then
>>> dma_map_sg() with the P1 struct device?
>>
>> Please don't do that.  dma_get_sgtable is a pretty broken API (see
>> the common near the arm implementation) and we should not add more
>> users of it.  If you want a piece of memory that can be mapped to
>> multiple devices allocate it using alloc_pages and then just map
>> it to each device.
> 
> Thanks for taking a look at this thread.
> 
> Unfortunately that wouldn't work. We have a specific reserved memory
> pool that is the only memory area accessible to one of the devices.
> Any idea how to handle this?

If it's reserved in the sense of being outside struct-page-backed 
"kernel memory", then provided you have a consistent CPU physical 
address it might be reasonable for other devices to access it via 
dma_map_resource().

Robin.

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:04                                 ` Robin Murphy
  0 siblings, 0 replies; 388+ messages in thread
From: Robin Murphy @ 2019-07-26 11:04 UTC (permalink / raw)
  To: Tomasz Figa, Christoph Hellwig
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS, Joerg Roedel, linux-arm-kernel,
	Matthias Brugger

On 26/07/2019 08:42, Tomasz Figa wrote:
> On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>>
>> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
>>> Could you try dma_get_sgtable() with the SCP struct device and then
>>> dma_map_sg() with the P1 struct device?
>>
>> Please don't do that.  dma_get_sgtable is a pretty broken API (see
>> the common near the arm implementation) and we should not add more
>> users of it.  If you want a piece of memory that can be mapped to
>> multiple devices allocate it using alloc_pages and then just map
>> it to each device.
> 
> Thanks for taking a look at this thread.
> 
> Unfortunately that wouldn't work. We have a specific reserved memory
> pool that is the only memory area accessible to one of the devices.
> Any idea how to handle this?

If it's reserved in the sense of being outside struct-page-backed 
"kernel memory", then provided you have a consistent CPU physical 
address it might be reasonable for other devices to access it via 
dma_map_resource().

Robin.

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:04                                 ` Robin Murphy
  0 siblings, 0 replies; 388+ messages in thread
From: Robin Murphy @ 2019-07-26 11:04 UTC (permalink / raw)
  To: Tomasz Figa, Christoph Hellwig
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Matthias Brugger, Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Mauro Carvalho Chehab, linux-arm-kernel,
	Linux Media Mailing List

On 26/07/2019 08:42, Tomasz Figa wrote:
> On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>>
>> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
>>> Could you try dma_get_sgtable() with the SCP struct device and then
>>> dma_map_sg() with the P1 struct device?
>>
>> Please don't do that.  dma_get_sgtable is a pretty broken API (see
>> the common near the arm implementation) and we should not add more
>> users of it.  If you want a piece of memory that can be mapped to
>> multiple devices allocate it using alloc_pages and then just map
>> it to each device.
> 
> Thanks for taking a look at this thread.
> 
> Unfortunately that wouldn't work. We have a specific reserved memory
> pool that is the only memory area accessible to one of the devices.
> Any idea how to handle this?

If it's reserved in the sense of being outside struct-page-backed 
"kernel memory", then provided you have a consistent CPU physical 
address it might be reasonable for other devices to access it via 
dma_map_resource().

Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:04                                 ` Robin Murphy
  0 siblings, 0 replies; 388+ messages in thread
From: Robin Murphy @ 2019-07-26 11:04 UTC (permalink / raw)
  To: Tomasz Figa, Christoph Hellwig
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Joerg Roedel,
	Ryan Yu (余孟修),
	Matthias Brugger, Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Hans Verkuil, Jungo Lin, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Mauro Carvalho Chehab, linux-arm-kernel,
	Linux Media Mailing List

On 26/07/2019 08:42, Tomasz Figa wrote:
> On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
>>
>> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
>>> Could you try dma_get_sgtable() with the SCP struct device and then
>>> dma_map_sg() with the P1 struct device?
>>
>> Please don't do that.  dma_get_sgtable is a pretty broken API (see
>> the common near the arm implementation) and we should not add more
>> users of it.  If you want a piece of memory that can be mapped to
>> multiple devices allocate it using alloc_pages and then just map
>> it to each device.
> 
> Thanks for taking a look at this thread.
> 
> Unfortunately that wouldn't work. We have a specific reserved memory
> pool that is the only memory area accessible to one of the devices.
> Any idea how to handle this?

If it's reserved in the sense of being outside struct-page-backed 
"kernel memory", then provided you have a consistent CPU physical 
address it might be reasonable for other devices to access it via 
dma_map_resource().

Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-26 11:04                                 ` Robin Murphy
  (?)
  (?)
@ 2019-07-26 11:59                                     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26 11:59 UTC (permalink / raw)
  To: Robin Murphy
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list-Y9sIeH5OGRo@public.gmane.org:IOMMU DRIVERS,
	Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Hans Verkuil, ddavenport-F7+t8E8rja9g9hUCZPvPmw

Hi Robin:

On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> On 26/07/2019 08:42, Tomasz Figa wrote:
> > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
> >>
> >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> >>> Could you try dma_get_sgtable() with the SCP struct device and then
> >>> dma_map_sg() with the P1 struct device?
> >>
> >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> >> the common near the arm implementation) and we should not add more
> >> users of it.  If you want a piece of memory that can be mapped to
> >> multiple devices allocate it using alloc_pages and then just map
> >> it to each device.
> > 
> > Thanks for taking a look at this thread.
> > 
> > Unfortunately that wouldn't work. We have a specific reserved memory
> > pool that is the only memory area accessible to one of the devices.
> > Any idea how to handle this?
> 
> If it's reserved in the sense of being outside struct-page-backed 
> "kernel memory", then provided you have a consistent CPU physical 
> address it might be reasonable for other devices to access it via 
> dma_map_resource().
> 
> Robin.

Thank you for your suggestion.

After revising to use dma_map_resource(), it is worked. Below is the
current implementation. Pleas kindly help us to check if there is any
misunderstanding.

#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000

	/*
	 * Allocate coherent reserved memory for SCP firmware usage.
	 * The size of SCP composer's memory is fixed to 0x200000
	 * for the requirement of firmware.
	 */
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/*
	 * This reserved memory is also be used by ISP P1 HW.
	 * Need to get iova address for ISP P1 DMA.
	 */
	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, appropriate Tomasz & Christoph's help on this issue.

Best regards,

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:59                                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26 11:59 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Tomasz Figa, Christoph Hellwig, devicetree,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, ddavenport,
	Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS, Joerg Roedel, linux-arm-kernel,
	Matthias Brugger

Hi Robin:

On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> On 26/07/2019 08:42, Tomasz Figa wrote:
> > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> >>
> >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> >>> Could you try dma_get_sgtable() with the SCP struct device and then
> >>> dma_map_sg() with the P1 struct device?
> >>
> >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> >> the common near the arm implementation) and we should not add more
> >> users of it.  If you want a piece of memory that can be mapped to
> >> multiple devices allocate it using alloc_pages and then just map
> >> it to each device.
> > 
> > Thanks for taking a look at this thread.
> > 
> > Unfortunately that wouldn't work. We have a specific reserved memory
> > pool that is the only memory area accessible to one of the devices.
> > Any idea how to handle this?
> 
> If it's reserved in the sense of being outside struct-page-backed 
> "kernel memory", then provided you have a consistent CPU physical 
> address it might be reasonable for other devices to access it via 
> dma_map_resource().
> 
> Robin.

Thank you for your suggestion.

After revising to use dma_map_resource(), it is worked. Below is the
current implementation. Pleas kindly help us to check if there is any
misunderstanding.

#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000

	/*
	 * Allocate coherent reserved memory for SCP firmware usage.
	 * The size of SCP composer's memory is fixed to 0x200000
	 * for the requirement of firmware.
	 */
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/*
	 * This reserved memory is also be used by ISP P1 HW.
	 * Need to get iova address for ISP P1 DMA.
	 */
	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, appropriate Tomasz & Christoph's help on this issue.

Best regards,

Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:59                                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26 11:59 UTC (permalink / raw)
  To: Robin Murphy
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Hans Verkuil, ddavenport, Mauro Carvalho Chehab,
	linux-arm-kernel, Linux Media Mailing List

Hi Robin:

On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> On 26/07/2019 08:42, Tomasz Figa wrote:
> > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> >>
> >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> >>> Could you try dma_get_sgtable() with the SCP struct device and then
> >>> dma_map_sg() with the P1 struct device?
> >>
> >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> >> the common near the arm implementation) and we should not add more
> >> users of it.  If you want a piece of memory that can be mapped to
> >> multiple devices allocate it using alloc_pages and then just map
> >> it to each device.
> > 
> > Thanks for taking a look at this thread.
> > 
> > Unfortunately that wouldn't work. We have a specific reserved memory
> > pool that is the only memory area accessible to one of the devices.
> > Any idea how to handle this?
> 
> If it's reserved in the sense of being outside struct-page-backed 
> "kernel memory", then provided you have a consistent CPU physical 
> address it might be reasonable for other devices to access it via 
> dma_map_resource().
> 
> Robin.

Thank you for your suggestion.

After revising to use dma_map_resource(), it is worked. Below is the
current implementation. Pleas kindly help us to check if there is any
misunderstanding.

#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000

	/*
	 * Allocate coherent reserved memory for SCP firmware usage.
	 * The size of SCP composer's memory is fixed to 0x200000
	 * for the requirement of firmware.
	 */
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/*
	 * This reserved memory is also be used by ISP P1 HW.
	 * Need to get iova address for ISP P1 DMA.
	 */
	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, appropriate Tomasz & Christoph's help on this issue.

Best regards,

Jungo

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 11:59                                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-26 11:59 UTC (permalink / raw)
  To: Robin Murphy
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Joerg Roedel,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Tomasz Figa, Christoph Hellwig,
	Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Hans Verkuil, ddavenport, Mauro Carvalho Chehab,
	linux-arm-kernel, Linux Media Mailing List

Hi Robin:

On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> On 26/07/2019 08:42, Tomasz Figa wrote:
> > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> >>
> >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> >>> Could you try dma_get_sgtable() with the SCP struct device and then
> >>> dma_map_sg() with the P1 struct device?
> >>
> >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> >> the common near the arm implementation) and we should not add more
> >> users of it.  If you want a piece of memory that can be mapped to
> >> multiple devices allocate it using alloc_pages and then just map
> >> it to each device.
> > 
> > Thanks for taking a look at this thread.
> > 
> > Unfortunately that wouldn't work. We have a specific reserved memory
> > pool that is the only memory area accessible to one of the devices.
> > Any idea how to handle this?
> 
> If it's reserved in the sense of being outside struct-page-backed 
> "kernel memory", then provided you have a consistent CPU physical 
> address it might be reasonable for other devices to access it via 
> dma_map_resource().
> 
> Robin.

Thank you for your suggestion.

After revising to use dma_map_resource(), it is worked. Below is the
current implementation. Pleas kindly help us to check if there is any
misunderstanding.

#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000

	/*
	 * Allocate coherent reserved memory for SCP firmware usage.
	 * The size of SCP composer's memory is fixed to 0x200000
	 * for the requirement of firmware.
	 */
	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
	if (!ptr) {
		dev_err(dev, "failed to allocate compose memory\n");
		return -ENOMEM;
	}
	p1_dev->composer_scp_addr = addr;
	p1_dev->composer_virt_addr = ptr;
	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);

	/*
	 * This reserved memory is also be used by ISP P1 HW.
	 * Need to get iova address for ISP P1 DMA.
	 */
	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
	if (dma_mapping_error(dev, addr)) {
		dev_err(dev, "Failed to map scp iova\n");
		ret = -ENOMEM;
		goto fail_free_mem;
	}
	p1_dev->composer_iova = addr;
	dev_info(dev, "scp iova addr:%pad\n", &addr);

Moreover, appropriate Tomasz & Christoph's help on this issue.

Best regards,

Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
  2019-07-26 11:59                                     ` Jungo Lin
  (?)
  (?)
@ 2019-07-26 14:04                                       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26 14:04 UTC (permalink / raw)
  To: Jungo Lin, Robin Murphy
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list-Y9sIeH5OGRo@public.gmane.org:IOMMU DRIVERS,
	Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Hans Verkuil, ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Mauro Carvalho Chehab, list-Y9sIeH5OGRo

On Fri, Jul 26, 2019 at 8:59 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
>
> Hi Robin:
>
> On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> > On 26/07/2019 08:42, Tomasz Figa wrote:
> > > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
> > >>
> > >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > >>> Could you try dma_get_sgtable() with the SCP struct device and then
> > >>> dma_map_sg() with the P1 struct device?
> > >>
> > >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> > >> the common near the arm implementation) and we should not add more
> > >> users of it.  If you want a piece of memory that can be mapped to
> > >> multiple devices allocate it using alloc_pages and then just map
> > >> it to each device.
> > >
> > > Thanks for taking a look at this thread.
> > >
> > > Unfortunately that wouldn't work. We have a specific reserved memory
> > > pool that is the only memory area accessible to one of the devices.
> > > Any idea how to handle this?
> >
> > If it's reserved in the sense of being outside struct-page-backed
> > "kernel memory", then provided you have a consistent CPU physical
> > address it might be reasonable for other devices to access it via
> > dma_map_resource().
> >
> > Robin.
>
> Thank you for your suggestion.
>
> After revising to use dma_map_resource(), it is worked. Below is the
> current implementation. Pleas kindly help us to check if there is any
> misunderstanding.
>
> #define MTK_ISP_COMPOSER_MEM_SIZE               0x200000
>
>         /*
>          * Allocate coherent reserved memory for SCP firmware usage.
>          * The size of SCP composer's memory is fixed to 0x200000
>          * for the requirement of firmware.
>          */
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /*
>          * This reserved memory is also be used by ISP P1 HW.
>          * Need to get iova address for ISP P1 DMA.
>          */
>         addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>                                 DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);

This is still incorrect, because addr is a DMA address, but the second
argument to dma_map_resource() is a physical address.

>         if (dma_mapping_error(dev, addr)) {
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, appropriate Tomasz & Christoph's help on this issue.

Robin, the memory is specified using the reserved-memory DT binding
and managed by the coherent DMA pool framework. We can allocate from
it using dma_alloc_coherent(), which gives us a DMA address, not CPU
physial address (although in practice on this platform they are equal
numerically).

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 14:04                                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26 14:04 UTC (permalink / raw)
  To: Jungo Lin, Robin Murphy
  Cc: Christoph Hellwig, devicetree,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	Linux Media Mailing List, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, ddavenport,
	Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS, Joerg Roedel,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Matthias Brugger

On Fri, Jul 26, 2019 at 8:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Robin:
>
> On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> > On 26/07/2019 08:42, Tomasz Figa wrote:
> > > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> > >>
> > >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > >>> Could you try dma_get_sgtable() with the SCP struct device and then
> > >>> dma_map_sg() with the P1 struct device?
> > >>
> > >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> > >> the common near the arm implementation) and we should not add more
> > >> users of it.  If you want a piece of memory that can be mapped to
> > >> multiple devices allocate it using alloc_pages and then just map
> > >> it to each device.
> > >
> > > Thanks for taking a look at this thread.
> > >
> > > Unfortunately that wouldn't work. We have a specific reserved memory
> > > pool that is the only memory area accessible to one of the devices.
> > > Any idea how to handle this?
> >
> > If it's reserved in the sense of being outside struct-page-backed
> > "kernel memory", then provided you have a consistent CPU physical
> > address it might be reasonable for other devices to access it via
> > dma_map_resource().
> >
> > Robin.
>
> Thank you for your suggestion.
>
> After revising to use dma_map_resource(), it is worked. Below is the
> current implementation. Pleas kindly help us to check if there is any
> misunderstanding.
>
> #define MTK_ISP_COMPOSER_MEM_SIZE               0x200000
>
>         /*
>          * Allocate coherent reserved memory for SCP firmware usage.
>          * The size of SCP composer's memory is fixed to 0x200000
>          * for the requirement of firmware.
>          */
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /*
>          * This reserved memory is also be used by ISP P1 HW.
>          * Need to get iova address for ISP P1 DMA.
>          */
>         addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>                                 DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);

This is still incorrect, because addr is a DMA address, but the second
argument to dma_map_resource() is a physical address.

>         if (dma_mapping_error(dev, addr)) {
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, appropriate Tomasz & Christoph's help on this issue.

Robin, the memory is specified using the reserved-memory DT binding
and managed by the coherent DMA pool framework. We can allocate from
it using dma_alloc_coherent(), which gives us a DMA address, not CPU
physial address (although in practice on this platform they are equal
numerically).

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 14:04                                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26 14:04 UTC (permalink / raw)
  To: Jungo Lin, Robin Murphy
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Hans Verkuil, ddavenport, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Fri, Jul 26, 2019 at 8:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Robin:
>
> On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> > On 26/07/2019 08:42, Tomasz Figa wrote:
> > > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> > >>
> > >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > >>> Could you try dma_get_sgtable() with the SCP struct device and then
> > >>> dma_map_sg() with the P1 struct device?
> > >>
> > >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> > >> the common near the arm implementation) and we should not add more
> > >> users of it.  If you want a piece of memory that can be mapped to
> > >> multiple devices allocate it using alloc_pages and then just map
> > >> it to each device.
> > >
> > > Thanks for taking a look at this thread.
> > >
> > > Unfortunately that wouldn't work. We have a specific reserved memory
> > > pool that is the only memory area accessible to one of the devices.
> > > Any idea how to handle this?
> >
> > If it's reserved in the sense of being outside struct-page-backed
> > "kernel memory", then provided you have a consistent CPU physical
> > address it might be reasonable for other devices to access it via
> > dma_map_resource().
> >
> > Robin.
>
> Thank you for your suggestion.
>
> After revising to use dma_map_resource(), it is worked. Below is the
> current implementation. Pleas kindly help us to check if there is any
> misunderstanding.
>
> #define MTK_ISP_COMPOSER_MEM_SIZE               0x200000
>
>         /*
>          * Allocate coherent reserved memory for SCP firmware usage.
>          * The size of SCP composer's memory is fixed to 0x200000
>          * for the requirement of firmware.
>          */
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /*
>          * This reserved memory is also be used by ISP P1 HW.
>          * Need to get iova address for ISP P1 DMA.
>          */
>         addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>                                 DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);

This is still incorrect, because addr is a DMA address, but the second
argument to dma_map_resource() is a physical address.

>         if (dma_mapping_error(dev, addr)) {
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, appropriate Tomasz & Christoph's help on this issue.

Robin, the memory is specified using the reserved-memory DT binding
and managed by the coherent DMA pool framework. We can allocate from
it using dma_alloc_coherent(), which gives us a DMA address, not CPU
physial address (although in practice on this platform they are equal
numerically).

Best regards,
Tomasz
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device
@ 2019-07-26 14:04                                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-26 14:04 UTC (permalink / raw)
  To: Jungo Lin, Robin Murphy
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Joerg Roedel,
	Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	list@263.net:IOMMU DRIVERS, Christoph Hellwig, Matthias Brugger,
	Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Hans Verkuil, ddavenport,
	Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Fri, Jul 26, 2019 at 8:59 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi Robin:
>
> On Fri, 2019-07-26 at 12:04 +0100, Robin Murphy wrote:
> > On 26/07/2019 08:42, Tomasz Figa wrote:
> > > On Fri, Jul 26, 2019 at 4:41 PM Christoph Hellwig <hch@infradead.org> wrote:
> > >>
> > >> On Fri, Jul 26, 2019 at 02:15:14PM +0900, Tomasz Figa wrote:
> > >>> Could you try dma_get_sgtable() with the SCP struct device and then
> > >>> dma_map_sg() with the P1 struct device?
> > >>
> > >> Please don't do that.  dma_get_sgtable is a pretty broken API (see
> > >> the common near the arm implementation) and we should not add more
> > >> users of it.  If you want a piece of memory that can be mapped to
> > >> multiple devices allocate it using alloc_pages and then just map
> > >> it to each device.
> > >
> > > Thanks for taking a look at this thread.
> > >
> > > Unfortunately that wouldn't work. We have a specific reserved memory
> > > pool that is the only memory area accessible to one of the devices.
> > > Any idea how to handle this?
> >
> > If it's reserved in the sense of being outside struct-page-backed
> > "kernel memory", then provided you have a consistent CPU physical
> > address it might be reasonable for other devices to access it via
> > dma_map_resource().
> >
> > Robin.
>
> Thank you for your suggestion.
>
> After revising to use dma_map_resource(), it is worked. Below is the
> current implementation. Pleas kindly help us to check if there is any
> misunderstanding.
>
> #define MTK_ISP_COMPOSER_MEM_SIZE               0x200000
>
>         /*
>          * Allocate coherent reserved memory for SCP firmware usage.
>          * The size of SCP composer's memory is fixed to 0x200000
>          * for the requirement of firmware.
>          */
>         ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>                                  MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>         if (!ptr) {
>                 dev_err(dev, "failed to allocate compose memory\n");
>                 return -ENOMEM;
>         }
>         p1_dev->composer_scp_addr = addr;
>         p1_dev->composer_virt_addr = ptr;
>         dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>
>         /*
>          * This reserved memory is also be used by ISP P1 HW.
>          * Need to get iova address for ISP P1 DMA.
>          */
>         addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>                                 DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);

This is still incorrect, because addr is a DMA address, but the second
argument to dma_map_resource() is a physical address.

>         if (dma_mapping_error(dev, addr)) {
>                 dev_err(dev, "Failed to map scp iova\n");
>                 ret = -ENOMEM;
>                 goto fail_free_mem;
>         }
>         p1_dev->composer_iova = addr;
>         dev_info(dev, "scp iova addr:%pad\n", &addr);
>
> Moreover, appropriate Tomasz & Christoph's help on this issue.

Robin, the memory is specified using the reserved-memory DT binding
and managed by the coherent DMA pool framework. We can allocate from
it using dma_alloc_coherent(), which gives us a DMA address, not CPU
physial address (although in practice on this platform they are equal
numerically).

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-26  5:49               ` Tomasz Figa
  (?)
@ 2019-07-29  1:18                 ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-29  1:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list

Hi, Tomasz:

On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > > [snip]
> > > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > > +                                 struct media_request *new_req)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +
> > > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > > +
> > > > > > +   if (!cam_dev->streaming) {
> > > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Normal enqueue flow */
> > > > > > +   if (new_req) {
> > > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Flush all media requests wehen first stream on */
> > > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > > +           list_del(&req->list);
> > > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > > +   }
> > > > > > +}
> > > > >
> > > > > This will have to be redone, as per the other suggestions, but generally one
> > > > > would have a function that tries to queue as much as possible from a list to
> > > > > the hardware and another function that adds a request to the list and calls
> > > > > the first function.
> > > > >
> > > >
> > > > We revised this function as below.
> > > > First to check the en-queue conditions:
> > > > a. stream on
> > > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > > at the same time.
> > > >
> > > >
> > > > Second, try to en-queue the frames in the pending job if possible and
> > > > move them into running job list if possible.
> > > >
> > > > The request has been inserted into pending job in mtk_cam_req_validate
> > > > which is used to validate media_request.
> > >
> > > Thanks for replying to each of the comments, that's very helpful.
> > > Snipped out the parts that I agreed with.
> > >
> > > Please note that req_validate is not supposed to change any driver
> > > state. It's only supposed to validate the request. req_queue is the
> > > right callback to insert the request into some internal driver
> > > bookkeeping structures.
> > >
> >
> > Yes, in req_validate function, we don't change any driver state.
> > Below is the function's implementation.
> >
> > a. Call vb2_request_validate(req) to verify media request.
> > b. Update the buffer internal structure buffer.
> > c. Insert the request into pending_job_list to prepare en-queue.
> >
> 
> Adding to a list is changing driver state. The callback must not
> modify anything else than the request itself.
> 
> Queuing to driver's list should happen in req_queue instead.
> 
> [snip]

Ok, got your point. We will move these implementation to .req_queue.

static const struct media_device_ops mtk_cam_media_ops = {
	.link_notify = v4l2_pipeline_link_notify,
	.req_alloc = mtk_cam_req_alloc,
	.req_free = mtk_cam_req_free,
	.req_validate = vb2_request_validate,
	.req_queue = mtk_cam_req_queue,
};

static void mtk_cam_req_queue(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);

	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irq(&cam->pending_job_lock);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irq(&cam->pending_job_lock);

	vb2_request_queue(req);
}

> > > >
> > > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > > {
> > > >         struct mtk_cam_dev_request *req, *req_prev;
> > > >         struct list_head enqueue_job_list;
> > > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > > >         unsigned long flags;
> > > >
> > > >         if (!cam_dev->streaming ||
> > > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> > >
> > > Do we have a guarantee that cam_dev->running_job_count doesn't
> > > decrement between the atomic_read() above and this line?
> > >
> >
> > Ok, we will use cam->pending_job_lock to protect
> > cam_dev->running_job_count access. Below is the revised version.
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> If we use a spin_lock to protect the counter, perhaps we don't need
> the atomic type anymore?
> 

Ok, we will remove atomic type usage.


> >                 dev_dbg(cam->dev, "jobs are full\n");
> >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >                 return;
> >         }
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> 
> Could we instead check the counter here and break if it's >=
> MTK_ISP_MAX_RUNNING_JOBS?
> Then we could increment it here too to simplify the code.
> 

Thanks for your advice.
We simplified this function as below:

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irq(&cam->pending_job_lock);
	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
			dev_dbg(cam->dev, "jobs are full\n");
			break;
		}
		cam->running_job_count++;
		list_del(&req->list);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
	spin_unlock_irq(&cam->pending_job_lock);
}

> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam->running_job_list);
> >                 mtk_isp_req_enqueue(cam, req);
> >                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >                 if (atomic_inc_return(&cam->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> 
> With the above suggestion, this if block would go away.
> 
> [snip]

Ditto.

> > > >                 mtk_isp_req_enqueue(cam_dev, req);
> > > >         }
> > > > }
> > > >
> > > [snip]
> > > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > > +
> > > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > > +           stride = ALIGN(stride, 4);
> > > > >
> > > > > Is it expected that only the F10 format needs this alignment?
> > > > >
> > > >
> > > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > > comments.
> > >
> > > That means that the B10 format also needs the extra alignment, as
> > > opposed to what the original code did, right?
> > >
> >
> > Sorry for short code snippet.
> > This alignment checking is only applied to F10, no B10.
> > If you like to check the full function, you could check this in this
> > link[1].
> >
> > static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> > node_id,
> >                              struct v4l2_pix_format_mplane *mp)
> > {
> >         unsigned int bpl, ppl;
> >         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >         unsigned int width = mp->width;
> >
> >         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> >                 /* bayer encoding format & 2 bytes alignment */
> >                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> >         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> >                 /*
> >                  * The FULL-G encoding format
> >                  * 1 G component per pixel
> >                  * 1 R component per 4 pixel
> >                  * 1 B component per 4 pixel
> >                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> >                  */
> >                 ppl = DIV_ROUND_UP(width * 6, 4);
> >                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> >
> >                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
> >                 if (pixel_bits == 10)
> >                         bpl = ALIGN(bpl, 4);
> >                 else
> >                         bpl = ALIGN(bpl, 8);
> >         }
> >
> > [1]
> > https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
> >
> 
> Got it, thanks!
> 
> [snip]
> > > > > > +
> > > > > > +static struct v4l2_subdev *
> > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > +{
> > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > +   struct media_entity *entity;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +   struct v4l2_subdev *sensor;
> > > > >
> > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > there no compiler warning generated for this?
> > > > >
> > > >
> > > > No, there is no compiler warning.
> > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > with different compiler options.
> > > >
> > >
> > > Thanks. It would be useful if you could check why the compiler you're
> > > using doesn't show a warning here. We might be missing other
> > > uninitialized variables.
> > >
> >
> > We will feedback to your project team to check the possible reason about
> > compiler warning issue.
> >
> 
> Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> emerge chromeos-kernel-4_19)?

> [snip]

Yes, I checked this comment in the Chromium OS build environment.
But, I think I have made the mistake here. I need to check the build
status in the Mediatek's kernel upstream environment. I will pay
attention in next path set upstream.

> > > > > > +
> > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > +           __func__,
> > > > > > +           node->id,
> > > > > > +           buf->vbb.request_fd,
> > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > +
> > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > +           return;
> > > > >
> > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > deal with two modes.
> > > > >
> > > >
> > > > The purpose of non-request function in this driver is needed by
> > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > request mode. Based on this requirement, do you have any suggestion?
> > > > For upstream driver, should we only consider request mode?
> > > >
> > >
> > > Where does that requirement come from? Why the timing of queuing of
> > > the buffers to the driver is important?
> > >
> > > [snip]
> >
> > Basically, this requirement comes from our internal camera
> > middle-ware/3A hal in user space. Since this is not generic requirement,
> > we will follow your original suggestion to keep the request mode only
> > and remove other non-request design in other files. For upstream driver,
> > it should support request mode only.
> >
> 
> Note that Chromium OS will use the "upstream driver" and we don't want
> to diverge, so please make the userspace also use only requests. I
> don't see a reason why there would be any need to submit any buffers
> outside of a request.
> 
> [snip]

Ok, I have raised your concern to our colleagues and let him to discuss
with you in another communication channel. 

> > > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > > +
> > > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > > >
> > > > > This would end up being called multiple times, once for each video node.
> > > > > Instead, this should be called explicitly by the driver when it completed
> > > > > the request - perhaps in the frame completion handler?
> > > > >
> > > > > With that, we probably wouldn't even need this callback.
> > > > >
> > > >
> > > > First, if we don't implement this callback function, we will receive
> > > > kernel warning as below.
> > > >
> > > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > > >
> > > > Second, this function is only be called in __vb2_queue_cancel function.
> > > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > > So could we just implement dummy empty function?
> > > >
> > > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > > but is
> > > >  *                      associated with a queued request was canceled.
> > > >  *                      The driver will have to mark associated objects in the
> > > >  *                      request as completed; required if requests are
> > > >  *                      supported.
> > > >
> > >
> > > Good catch, thanks.
> > >
> > > Sounds like we may indeed need to implement this callback. In
> > > particular, we may need to remove the request that the buffer was
> > > associated with from the driver queue and return the other buffers
> > > associated to it with an error state. This should be similar to
> > > handling a request failure.
> > > [snip]
> >
> > Before calling this callback function, the VB2's stop_streaming has been
> > called. Normally, we will return the buffers belonged to this vb2 queu
> > with error state. On other hand, only if the state of request is
> > MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> > __vb2_queue_cancel function. It hints this media request has been
> > validated and inserted into our driver's pending_job_list or
> > running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> > requests from driver's list when streaming is off. Since we have no
> > v4l2_ctrl, do we need to do the above things which is already handled in
> > mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> > only designed for v4l2_ctrl_request_complete usage?
> 
> Are you sure that this callback can be only called after
> stop_streaming? Also wouldn't that be after stop_streaming only on 1
> queue? The other queues could still remain streaming, but we still
> have to return corresponding buffers I believe.
> 
> Hans, could you clarify what exactly this callback is supposed to do?
> 

Ok, we will look forward Hans' comments on this.

> >
> > static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         dev_dbg(cam->dev, "%s\n", __func__);
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > }
> >
> > static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > {
> >         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >         struct device *dev = cam->dev;
> >
> >         dev_dbg(dev, "%s node:%d count info:%d", __func__,
> >                 node->id, atomic_read(&cam->stream_count));
> >
> >         mutex_lock(&cam->op_lock);
> >         if (atomic_read(&cam->stream_count) == cam->enabled_count)
> >                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
> >                         dev_err(dev, "failed to stop streaming\n");
> >
> >         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >
> >         /* Check the first node to stream-off */
> >         if (!atomic_dec_and_test(&cam->stream_count)) {
> >                 mutex_unlock(&cam->op_lock);
> >                 return;
> >         }
> >         mutex_unlock(&cam->op_lock);
> >
> >         mtk_cam_dev_req_cleanup(cam);
> >         media_pipeline_stop(&node->vdev.entity);
> > }
> 
> [keeping the context for Hans]
> 
> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


Best regards,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-29  1:18                 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-29  1:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Hans Verkuil, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > > [snip]
> > > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > > +                                 struct media_request *new_req)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +
> > > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > > +
> > > > > > +   if (!cam_dev->streaming) {
> > > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Normal enqueue flow */
> > > > > > +   if (new_req) {
> > > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Flush all media requests wehen first stream on */
> > > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > > +           list_del(&req->list);
> > > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > > +   }
> > > > > > +}
> > > > >
> > > > > This will have to be redone, as per the other suggestions, but generally one
> > > > > would have a function that tries to queue as much as possible from a list to
> > > > > the hardware and another function that adds a request to the list and calls
> > > > > the first function.
> > > > >
> > > >
> > > > We revised this function as below.
> > > > First to check the en-queue conditions:
> > > > a. stream on
> > > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > > at the same time.
> > > >
> > > >
> > > > Second, try to en-queue the frames in the pending job if possible and
> > > > move them into running job list if possible.
> > > >
> > > > The request has been inserted into pending job in mtk_cam_req_validate
> > > > which is used to validate media_request.
> > >
> > > Thanks for replying to each of the comments, that's very helpful.
> > > Snipped out the parts that I agreed with.
> > >
> > > Please note that req_validate is not supposed to change any driver
> > > state. It's only supposed to validate the request. req_queue is the
> > > right callback to insert the request into some internal driver
> > > bookkeeping structures.
> > >
> >
> > Yes, in req_validate function, we don't change any driver state.
> > Below is the function's implementation.
> >
> > a. Call vb2_request_validate(req) to verify media request.
> > b. Update the buffer internal structure buffer.
> > c. Insert the request into pending_job_list to prepare en-queue.
> >
> 
> Adding to a list is changing driver state. The callback must not
> modify anything else than the request itself.
> 
> Queuing to driver's list should happen in req_queue instead.
> 
> [snip]

Ok, got your point. We will move these implementation to .req_queue.

static const struct media_device_ops mtk_cam_media_ops = {
	.link_notify = v4l2_pipeline_link_notify,
	.req_alloc = mtk_cam_req_alloc,
	.req_free = mtk_cam_req_free,
	.req_validate = vb2_request_validate,
	.req_queue = mtk_cam_req_queue,
};

static void mtk_cam_req_queue(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);

	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irq(&cam->pending_job_lock);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irq(&cam->pending_job_lock);

	vb2_request_queue(req);
}

> > > >
> > > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > > {
> > > >         struct mtk_cam_dev_request *req, *req_prev;
> > > >         struct list_head enqueue_job_list;
> > > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > > >         unsigned long flags;
> > > >
> > > >         if (!cam_dev->streaming ||
> > > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> > >
> > > Do we have a guarantee that cam_dev->running_job_count doesn't
> > > decrement between the atomic_read() above and this line?
> > >
> >
> > Ok, we will use cam->pending_job_lock to protect
> > cam_dev->running_job_count access. Below is the revised version.
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> If we use a spin_lock to protect the counter, perhaps we don't need
> the atomic type anymore?
> 

Ok, we will remove atomic type usage.


> >                 dev_dbg(cam->dev, "jobs are full\n");
> >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >                 return;
> >         }
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> 
> Could we instead check the counter here and break if it's >=
> MTK_ISP_MAX_RUNNING_JOBS?
> Then we could increment it here too to simplify the code.
> 

Thanks for your advice.
We simplified this function as below:

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irq(&cam->pending_job_lock);
	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
			dev_dbg(cam->dev, "jobs are full\n");
			break;
		}
		cam->running_job_count++;
		list_del(&req->list);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
	spin_unlock_irq(&cam->pending_job_lock);
}

> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam->running_job_list);
> >                 mtk_isp_req_enqueue(cam, req);
> >                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >                 if (atomic_inc_return(&cam->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> 
> With the above suggestion, this if block would go away.
> 
> [snip]

Ditto.

> > > >                 mtk_isp_req_enqueue(cam_dev, req);
> > > >         }
> > > > }
> > > >
> > > [snip]
> > > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > > +
> > > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > > +           stride = ALIGN(stride, 4);
> > > > >
> > > > > Is it expected that only the F10 format needs this alignment?
> > > > >
> > > >
> > > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > > comments.
> > >
> > > That means that the B10 format also needs the extra alignment, as
> > > opposed to what the original code did, right?
> > >
> >
> > Sorry for short code snippet.
> > This alignment checking is only applied to F10, no B10.
> > If you like to check the full function, you could check this in this
> > link[1].
> >
> > static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> > node_id,
> >                              struct v4l2_pix_format_mplane *mp)
> > {
> >         unsigned int bpl, ppl;
> >         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >         unsigned int width = mp->width;
> >
> >         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> >                 /* bayer encoding format & 2 bytes alignment */
> >                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> >         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> >                 /*
> >                  * The FULL-G encoding format
> >                  * 1 G component per pixel
> >                  * 1 R component per 4 pixel
> >                  * 1 B component per 4 pixel
> >                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> >                  */
> >                 ppl = DIV_ROUND_UP(width * 6, 4);
> >                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> >
> >                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
> >                 if (pixel_bits == 10)
> >                         bpl = ALIGN(bpl, 4);
> >                 else
> >                         bpl = ALIGN(bpl, 8);
> >         }
> >
> > [1]
> > https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
> >
> 
> Got it, thanks!
> 
> [snip]
> > > > > > +
> > > > > > +static struct v4l2_subdev *
> > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > +{
> > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > +   struct media_entity *entity;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +   struct v4l2_subdev *sensor;
> > > > >
> > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > there no compiler warning generated for this?
> > > > >
> > > >
> > > > No, there is no compiler warning.
> > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > with different compiler options.
> > > >
> > >
> > > Thanks. It would be useful if you could check why the compiler you're
> > > using doesn't show a warning here. We might be missing other
> > > uninitialized variables.
> > >
> >
> > We will feedback to your project team to check the possible reason about
> > compiler warning issue.
> >
> 
> Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> emerge chromeos-kernel-4_19)?

> [snip]

Yes, I checked this comment in the Chromium OS build environment.
But, I think I have made the mistake here. I need to check the build
status in the Mediatek's kernel upstream environment. I will pay
attention in next path set upstream.

> > > > > > +
> > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > +           __func__,
> > > > > > +           node->id,
> > > > > > +           buf->vbb.request_fd,
> > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > +
> > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > +           return;
> > > > >
> > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > deal with two modes.
> > > > >
> > > >
> > > > The purpose of non-request function in this driver is needed by
> > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > request mode. Based on this requirement, do you have any suggestion?
> > > > For upstream driver, should we only consider request mode?
> > > >
> > >
> > > Where does that requirement come from? Why the timing of queuing of
> > > the buffers to the driver is important?
> > >
> > > [snip]
> >
> > Basically, this requirement comes from our internal camera
> > middle-ware/3A hal in user space. Since this is not generic requirement,
> > we will follow your original suggestion to keep the request mode only
> > and remove other non-request design in other files. For upstream driver,
> > it should support request mode only.
> >
> 
> Note that Chromium OS will use the "upstream driver" and we don't want
> to diverge, so please make the userspace also use only requests. I
> don't see a reason why there would be any need to submit any buffers
> outside of a request.
> 
> [snip]

Ok, I have raised your concern to our colleagues and let him to discuss
with you in another communication channel. 

> > > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > > +
> > > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > > >
> > > > > This would end up being called multiple times, once for each video node.
> > > > > Instead, this should be called explicitly by the driver when it completed
> > > > > the request - perhaps in the frame completion handler?
> > > > >
> > > > > With that, we probably wouldn't even need this callback.
> > > > >
> > > >
> > > > First, if we don't implement this callback function, we will receive
> > > > kernel warning as below.
> > > >
> > > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > > >
> > > > Second, this function is only be called in __vb2_queue_cancel function.
> > > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > > So could we just implement dummy empty function?
> > > >
> > > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > > but is
> > > >  *                      associated with a queued request was canceled.
> > > >  *                      The driver will have to mark associated objects in the
> > > >  *                      request as completed; required if requests are
> > > >  *                      supported.
> > > >
> > >
> > > Good catch, thanks.
> > >
> > > Sounds like we may indeed need to implement this callback. In
> > > particular, we may need to remove the request that the buffer was
> > > associated with from the driver queue and return the other buffers
> > > associated to it with an error state. This should be similar to
> > > handling a request failure.
> > > [snip]
> >
> > Before calling this callback function, the VB2's stop_streaming has been
> > called. Normally, we will return the buffers belonged to this vb2 queu
> > with error state. On other hand, only if the state of request is
> > MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> > __vb2_queue_cancel function. It hints this media request has been
> > validated and inserted into our driver's pending_job_list or
> > running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> > requests from driver's list when streaming is off. Since we have no
> > v4l2_ctrl, do we need to do the above things which is already handled in
> > mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> > only designed for v4l2_ctrl_request_complete usage?
> 
> Are you sure that this callback can be only called after
> stop_streaming? Also wouldn't that be after stop_streaming only on 1
> queue? The other queues could still remain streaming, but we still
> have to return corresponding buffers I believe.
> 
> Hans, could you clarify what exactly this callback is supposed to do?
> 

Ok, we will look forward Hans' comments on this.

> >
> > static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         dev_dbg(cam->dev, "%s\n", __func__);
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > }
> >
> > static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > {
> >         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >         struct device *dev = cam->dev;
> >
> >         dev_dbg(dev, "%s node:%d count info:%d", __func__,
> >                 node->id, atomic_read(&cam->stream_count));
> >
> >         mutex_lock(&cam->op_lock);
> >         if (atomic_read(&cam->stream_count) == cam->enabled_count)
> >                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
> >                         dev_err(dev, "failed to stop streaming\n");
> >
> >         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >
> >         /* Check the first node to stream-off */
> >         if (!atomic_dec_and_test(&cam->stream_count)) {
> >                 mutex_unlock(&cam->op_lock);
> >                 return;
> >         }
> >         mutex_unlock(&cam->op_lock);
> >
> >         mtk_cam_dev_req_cleanup(cam);
> >         media_pipeline_stop(&node->vdev.entity);
> > }
> 
> [keeping the context for Hans]
> 
> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


Best regards,


Jungo




^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-29  1:18                 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-29  1:18 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > Hi Jungo,
> > >
> > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> > > [snip]
> > > > > > +static void mtk_cam_req_try_isp_queue(struct mtk_cam_dev *cam_dev,
> > > > > > +                                 struct media_request *new_req)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev_request *req, *req_safe, *cam_dev_req;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +
> > > > > > +   dev_dbg(dev, "%s new req:%d", __func__, !new_req);
> > > > > > +
> > > > > > +   if (!cam_dev->streaming) {
> > > > > > +           cam_dev_req = mtk_cam_req_to_dev_req(new_req);
> > > > > > +           spin_lock(&cam_dev->req_lock);
> > > > > > +           list_add_tail(&cam_dev_req->list, &cam_dev->req_list);
> > > > > > +           spin_unlock(&cam_dev->req_lock);
> > > > > > +           dev_dbg(dev, "%s: stream off, no ISP enqueue\n", __func__);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Normal enqueue flow */
> > > > > > +   if (new_req) {
> > > > > > +           mtk_isp_req_enqueue(dev, new_req);
> > > > > > +           return;
> > > > > > +   }
> > > > > > +
> > > > > > +   /* Flush all media requests wehen first stream on */
> > > > > > +   list_for_each_entry_safe(req, req_safe, &cam_dev->req_list, list) {
> > > > > > +           list_del(&req->list);
> > > > > > +           mtk_isp_req_enqueue(dev, &req->req);
> > > > > > +   }
> > > > > > +}
> > > > >
> > > > > This will have to be redone, as per the other suggestions, but generally one
> > > > > would have a function that tries to queue as much as possible from a list to
> > > > > the hardware and another function that adds a request to the list and calls
> > > > > the first function.
> > > > >
> > > >
> > > > We revised this function as below.
> > > > First to check the en-queue conditions:
> > > > a. stream on
> > > > b. The composer buffers in SCP are 3, so we only could has 3 jobs
> > > > at the same time.
> > > >
> > > >
> > > > Second, try to en-queue the frames in the pending job if possible and
> > > > move them into running job list if possible.
> > > >
> > > > The request has been inserted into pending job in mtk_cam_req_validate
> > > > which is used to validate media_request.
> > >
> > > Thanks for replying to each of the comments, that's very helpful.
> > > Snipped out the parts that I agreed with.
> > >
> > > Please note that req_validate is not supposed to change any driver
> > > state. It's only supposed to validate the request. req_queue is the
> > > right callback to insert the request into some internal driver
> > > bookkeeping structures.
> > >
> >
> > Yes, in req_validate function, we don't change any driver state.
> > Below is the function's implementation.
> >
> > a. Call vb2_request_validate(req) to verify media request.
> > b. Update the buffer internal structure buffer.
> > c. Insert the request into pending_job_list to prepare en-queue.
> >
> 
> Adding to a list is changing driver state. The callback must not
> modify anything else than the request itself.
> 
> Queuing to driver's list should happen in req_queue instead.
> 
> [snip]

Ok, got your point. We will move these implementation to .req_queue.

static const struct media_device_ops mtk_cam_media_ops = {
	.link_notify = v4l2_pipeline_link_notify,
	.req_alloc = mtk_cam_req_alloc,
	.req_free = mtk_cam_req_free,
	.req_validate = vb2_request_validate,
	.req_queue = mtk_cam_req_queue,
};

static void mtk_cam_req_queue(struct media_request *req)
{
	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
					       media_dev);

	atomic_set(&cam_req->buf_count, vb2_request_buffer_cnt(req));

	/* add to pending job list */
	spin_lock_irq(&cam->pending_job_lock);
	list_add_tail(&cam_req->list, &cam->pending_job_list);
	spin_unlock_irq(&cam->pending_job_lock);

	vb2_request_queue(req);
}

> > > >
> > > > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev)
> > > > {
> > > >         struct mtk_cam_dev_request *req, *req_prev;
> > > >         struct list_head enqueue_job_list;
> > > >         int buffer_cnt = atomic_read(&cam_dev->running_job_count);
> > > >         unsigned long flags;
> > > >
> > > >         if (!cam_dev->streaming ||
> > > >             buffer_cnt >= MTK_ISP_MAX_RUNNING_JOBS) {
> > >
> > > Do we have a guarantee that cam_dev->running_job_count doesn't
> > > decrement between the atomic_read() above and this line?
> > >
> >
> > Ok, we will use cam->pending_job_lock to protect
> > cam_dev->running_job_count access. Below is the revised version.
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         if (atomic_read(&cam->running_job_count) >= MTK_ISP_MAX_RUNNING_JOBS) {
> 
> If we use a spin_lock to protect the counter, perhaps we don't need
> the atomic type anymore?
> 

Ok, we will remove atomic type usage.


> >                 dev_dbg(cam->dev, "jobs are full\n");
> >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >                 return;
> >         }
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> 
> Could we instead check the counter here and break if it's >=
> MTK_ISP_MAX_RUNNING_JOBS?
> Then we could increment it here too to simplify the code.
> 

Thanks for your advice.
We simplified this function as below:

void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
{
	struct mtk_cam_dev_request *req, *req_prev;
	unsigned long flags;

	if (!cam->streaming) {
		dev_dbg(cam->dev, "stream is off\n");
		return;
	}

	spin_lock_irq(&cam->pending_job_lock);
	spin_lock_irqsave(&cam->running_job_lock, flags);
	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
			dev_dbg(cam->dev, "jobs are full\n");
			break;
		}
		cam->running_job_count++;
		list_del(&req->list);
		list_add_tail(&req->list, &cam->running_job_list);
		mtk_isp_req_enqueue(cam, req);
	}
	spin_unlock_irqrestore(&cam->running_job_lock, flags);
	spin_unlock_irq(&cam->pending_job_lock);
}

> >                 list_del(&req->list);
> >                 spin_lock_irqsave(&cam->running_job_lock, flags);
> >                 list_add_tail(&req->list, &cam->running_job_list);
> >                 mtk_isp_req_enqueue(cam, req);
> >                 spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >                 if (atomic_inc_return(&cam->running_job_count) >=
> >                         MTK_ISP_MAX_RUNNING_JOBS)
> >                         break;
> 
> With the above suggestion, this if block would go away.
> 
> [snip]

Ditto.

> > > >                 mtk_isp_req_enqueue(cam_dev, req);
> > > >         }
> > > > }
> > > >
> > > [snip]
> > > > > > +   stride = DIV_ROUND_UP(stride * pixel_byte, 8);
> > > > > > +
> > > > > > +   if (pix_fmt == V4L2_PIX_FMT_MTISP_F10)
> > > > > > +           stride = ALIGN(stride, 4);
> > > > >
> > > > > Is it expected that only the F10 format needs this alignment?
> > > > >
> > > >
> > > > yes, if the pixel bits of image format is 10, the byte alignment of bpl
> > > > should be 4. Otherwise, it is 8. We will revise this and add more
> > > > comments.
> > >
> > > That means that the B10 format also needs the extra alignment, as
> > > opposed to what the original code did, right?
> > >
> >
> > Sorry for short code snippet.
> > This alignment checking is only applied to F10, no B10.
> > If you like to check the full function, you could check this in this
> > link[1].
> >
> > static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int
> > node_id,
> >                              struct v4l2_pix_format_mplane *mp)
> > {
> >         unsigned int bpl, ppl;
> >         unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >         unsigned int width = mp->width;
> >
> >         if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> >                 /* bayer encoding format & 2 bytes alignment */
> >                 bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> >         } else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> >                 /*
> >                  * The FULL-G encoding format
> >                  * 1 G component per pixel
> >                  * 1 R component per 4 pixel
> >                  * 1 B component per 4 pixel
> >                  * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> >                  */
> >                 ppl = DIV_ROUND_UP(width * 6, 4);
> >                 bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> >
> >                 /* 4 bytes alignment for 10 bit & others are 8 bytes */
> >                 if (pixel_bits == 10)
> >                         bpl = ALIGN(bpl, 4);
> >                 else
> >                         bpl = ALIGN(bpl, 8);
> >         }
> >
> > [1]
> > https://crrev.com/c/1712885/2/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c#303
> >
> 
> Got it, thanks!
> 
> [snip]
> > > > > > +
> > > > > > +static struct v4l2_subdev *
> > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > +{
> > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > +   struct media_entity *entity;
> > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > +   struct v4l2_subdev *sensor;
> > > > >
> > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > there no compiler warning generated for this?
> > > > >
> > > >
> > > > No, there is no compiler warning.
> > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > with different compiler options.
> > > >
> > >
> > > Thanks. It would be useful if you could check why the compiler you're
> > > using doesn't show a warning here. We might be missing other
> > > uninitialized variables.
> > >
> >
> > We will feedback to your project team to check the possible reason about
> > compiler warning issue.
> >
> 
> Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> emerge chromeos-kernel-4_19)?

> [snip]

Yes, I checked this comment in the Chromium OS build environment.
But, I think I have made the mistake here. I need to check the build
status in the Mediatek's kernel upstream environment. I will pay
attention in next path set upstream.

> > > > > > +
> > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > +           __func__,
> > > > > > +           node->id,
> > > > > > +           buf->vbb.request_fd,
> > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > +
> > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > +           return;
> > > > >
> > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > deal with two modes.
> > > > >
> > > >
> > > > The purpose of non-request function in this driver is needed by
> > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > request mode. Based on this requirement, do you have any suggestion?
> > > > For upstream driver, should we only consider request mode?
> > > >
> > >
> > > Where does that requirement come from? Why the timing of queuing of
> > > the buffers to the driver is important?
> > >
> > > [snip]
> >
> > Basically, this requirement comes from our internal camera
> > middle-ware/3A hal in user space. Since this is not generic requirement,
> > we will follow your original suggestion to keep the request mode only
> > and remove other non-request design in other files. For upstream driver,
> > it should support request mode only.
> >
> 
> Note that Chromium OS will use the "upstream driver" and we don't want
> to diverge, so please make the userspace also use only requests. I
> don't see a reason why there would be any need to submit any buffers
> outside of a request.
> 
> [snip]

Ok, I have raised your concern to our colleagues and let him to discuss
with you in another communication channel. 

> > > > > > +static void mtk_cam_vb2_buf_request_complete(struct vb2_buffer *vb)
> > > > > > +{
> > > > > > +   struct mtk_cam_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
> > > > > > +
> > > > > > +   v4l2_ctrl_request_complete(vb->req_obj.req,
> > > > > > +                              dev->v4l2_dev.ctrl_handler);
> > > > >
> > > > > This would end up being called multiple times, once for each video node.
> > > > > Instead, this should be called explicitly by the driver when it completed
> > > > > the request - perhaps in the frame completion handler?
> > > > >
> > > > > With that, we probably wouldn't even need this callback.
> > > > >
> > > >
> > > > First, if we don't implement this callback function, we will receive
> > > > kernel warning as below.
> > > >
> > > > https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L420
> > > >
> > > > Second, this function is only be called in __vb2_queue_cancel function.
> > > > Moreover, we will remove cam_dev->v4l2_dev.ctrl_handler in next patch.
> > > > So could we just implement dummy empty function?
> > > >
> > > >  * @buf_request_complete: a buffer that was never queued to the driver
> > > > but is
> > > >  *                      associated with a queued request was canceled.
> > > >  *                      The driver will have to mark associated objects in the
> > > >  *                      request as completed; required if requests are
> > > >  *                      supported.
> > > >
> > >
> > > Good catch, thanks.
> > >
> > > Sounds like we may indeed need to implement this callback. In
> > > particular, we may need to remove the request that the buffer was
> > > associated with from the driver queue and return the other buffers
> > > associated to it with an error state. This should be similar to
> > > handling a request failure.
> > > [snip]
> >
> > Before calling this callback function, the VB2's stop_streaming has been
> > called. Normally, we will return the buffers belonged to this vb2 queu
> > with error state. On other hand, only if the state of request is
> > MEDIA_REQUEST_STATE_QUEUED, the buf_request_complete will be called in
> > __vb2_queue_cancel function. It hints this media request has been
> > validated and inserted into our driver's pending_job_list or
> > running_job_list. So we will call mtk_cam_dev_req_cleanup() remove these
> > requests from driver's list when streaming is off. Since we have no
> > v4l2_ctrl, do we need to do the above things which is already handled in
> > mtk_cam_vb2_stop_streaming function? Maybe is this callback function
> > only designed for v4l2_ctrl_request_complete usage?
> 
> Are you sure that this callback can be only called after
> stop_streaming? Also wouldn't that be after stop_streaming only on 1
> queue? The other queues could still remain streaming, but we still
> have to return corresponding buffers I believe.
> 
> Hans, could you clarify what exactly this callback is supposed to do?
> 

Ok, we will look forward Hans' comments on this.

> >
> > static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         dev_dbg(cam->dev, "%s\n", __func__);
> >
> >         spin_lock_irqsave(&cam->pending_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> >
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> >         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> >                 list_del(&req->list);
> >         spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > }
> >
> > static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > {
> >         struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >         struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >         struct device *dev = cam->dev;
> >
> >         dev_dbg(dev, "%s node:%d count info:%d", __func__,
> >                 node->id, atomic_read(&cam->stream_count));
> >
> >         mutex_lock(&cam->op_lock);
> >         if (atomic_read(&cam->stream_count) == cam->enabled_count)
> >                 if (v4l2_subdev_call(&cam->subdev, video, s_stream, 0))
> >                         dev_err(dev, "failed to stop streaming\n");
> >
> >         mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >
> >         /* Check the first node to stream-off */
> >         if (!atomic_dec_and_test(&cam->stream_count)) {
> >                 mutex_unlock(&cam->op_lock);
> >                 return;
> >         }
> >         mutex_unlock(&cam->op_lock);
> >
> >         mtk_cam_dev_req_cleanup(cam);
> >         media_pipeline_stop(&node->vdev.entity);
> > }
> 
> [keeping the context for Hans]
> 
> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


Best regards,


Jungo




_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-29  1:18                 ` Jungo Lin
  (?)
@ 2019-07-29 10:04                   ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-29 10:04 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元)

On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > >                 dev_dbg(cam->dev, "jobs are full\n");
> > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > >                 return;
> > >         }
> > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> >
> > Could we instead check the counter here and break if it's >=
> > MTK_ISP_MAX_RUNNING_JOBS?
> > Then we could increment it here too to simplify the code.
> >
>
> Thanks for your advice.
> We simplified this function as below:
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irq(&cam->pending_job_lock);
>         spin_lock_irqsave(&cam->running_job_lock, flags);

Having the inner call spin_lock_irqsave() doesn't really do anything
useful, because the outer spin_lock_irq() disables the IRQs and flags
would always have the IRQ disabled state. Please use irqsave for the
outer call.

[snip]
> > > > > > > +
> > > > > > > +static struct v4l2_subdev *
> > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > +{
> > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > +   struct media_entity *entity;
> > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > +   struct v4l2_subdev *sensor;
> > > > > >
> > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > there no compiler warning generated for this?
> > > > > >
> > > > >
> > > > > No, there is no compiler warning.
> > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > with different compiler options.
> > > > >
> > > >
> > > > Thanks. It would be useful if you could check why the compiler you're
> > > > using doesn't show a warning here. We might be missing other
> > > > uninitialized variables.
> > > >
> > >
> > > We will feedback to your project team to check the possible reason about
> > > compiler warning issue.
> > >
> >
> > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > emerge chromeos-kernel-4_19)?
>
> > [snip]
>
> Yes, I checked this comment in the Chromium OS build environment.
> But, I think I have made the mistake here. I need to check the build
> status in the Mediatek's kernel upstream environment. I will pay
> attention in next path set upstream.
>

Thanks a lot. I will recheck this in the Chromium OS toolchain too.

> > > > > > > +
> > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > +           __func__,
> > > > > > > +           node->id,
> > > > > > > +           buf->vbb.request_fd,
> > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > +
> > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > +           return;
> > > > > >
> > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > deal with two modes.
> > > > > >
> > > > >
> > > > > The purpose of non-request function in this driver is needed by
> > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > For upstream driver, should we only consider request mode?
> > > > >
> > > >
> > > > Where does that requirement come from? Why the timing of queuing of
> > > > the buffers to the driver is important?
> > > >
> > > > [snip]
> > >
> > > Basically, this requirement comes from our internal camera
> > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > we will follow your original suggestion to keep the request mode only
> > > and remove other non-request design in other files. For upstream driver,
> > > it should support request mode only.
> > >
> >
> > Note that Chromium OS will use the "upstream driver" and we don't want
> > to diverge, so please make the userspace also use only requests. I
> > don't see a reason why there would be any need to submit any buffers
> > outside of a request.
> >
> > [snip]
>
> Ok, I have raised your concern to our colleagues and let him to discuss
> with you in another communication channel.
>

Thanks!

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-29 10:04                   ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-29 10:04 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > >                 dev_dbg(cam->dev, "jobs are full\n");
> > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > >                 return;
> > >         }
> > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> >
> > Could we instead check the counter here and break if it's >=
> > MTK_ISP_MAX_RUNNING_JOBS?
> > Then we could increment it here too to simplify the code.
> >
>
> Thanks for your advice.
> We simplified this function as below:
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irq(&cam->pending_job_lock);
>         spin_lock_irqsave(&cam->running_job_lock, flags);

Having the inner call spin_lock_irqsave() doesn't really do anything
useful, because the outer spin_lock_irq() disables the IRQs and flags
would always have the IRQ disabled state. Please use irqsave for the
outer call.

[snip]
> > > > > > > +
> > > > > > > +static struct v4l2_subdev *
> > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > +{
> > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > +   struct media_entity *entity;
> > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > +   struct v4l2_subdev *sensor;
> > > > > >
> > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > there no compiler warning generated for this?
> > > > > >
> > > > >
> > > > > No, there is no compiler warning.
> > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > with different compiler options.
> > > > >
> > > >
> > > > Thanks. It would be useful if you could check why the compiler you're
> > > > using doesn't show a warning here. We might be missing other
> > > > uninitialized variables.
> > > >
> > >
> > > We will feedback to your project team to check the possible reason about
> > > compiler warning issue.
> > >
> >
> > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > emerge chromeos-kernel-4_19)?
>
> > [snip]
>
> Yes, I checked this comment in the Chromium OS build environment.
> But, I think I have made the mistake here. I need to check the build
> status in the Mediatek's kernel upstream environment. I will pay
> attention in next path set upstream.
>

Thanks a lot. I will recheck this in the Chromium OS toolchain too.

> > > > > > > +
> > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > +           __func__,
> > > > > > > +           node->id,
> > > > > > > +           buf->vbb.request_fd,
> > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > +
> > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > +           return;
> > > > > >
> > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > deal with two modes.
> > > > > >
> > > > >
> > > > > The purpose of non-request function in this driver is needed by
> > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > For upstream driver, should we only consider request mode?
> > > > >
> > > >
> > > > Where does that requirement come from? Why the timing of queuing of
> > > > the buffers to the driver is important?
> > > >
> > > > [snip]
> > >
> > > Basically, this requirement comes from our internal camera
> > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > we will follow your original suggestion to keep the request mode only
> > > and remove other non-request design in other files. For upstream driver,
> > > it should support request mode only.
> > >
> >
> > Note that Chromium OS will use the "upstream driver" and we don't want
> > to diverge, so please make the userspace also use only requests. I
> > don't see a reason why there would be any need to submit any buffers
> > outside of a request.
> >
> > [snip]
>
> Ok, I have raised your concern to our colleagues and let him to discuss
> with you in another communication channel.
>

Thanks!

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-29 10:04                   ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-07-29 10:04 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > >                 dev_dbg(cam->dev, "jobs are full\n");
> > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > >                 return;
> > >         }
> > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> >
> > Could we instead check the counter here and break if it's >=
> > MTK_ISP_MAX_RUNNING_JOBS?
> > Then we could increment it here too to simplify the code.
> >
>
> Thanks for your advice.
> We simplified this function as below:
>
> void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         if (!cam->streaming) {
>                 dev_dbg(cam->dev, "stream is off\n");
>                 return;
>         }
>
>         spin_lock_irq(&cam->pending_job_lock);
>         spin_lock_irqsave(&cam->running_job_lock, flags);

Having the inner call spin_lock_irqsave() doesn't really do anything
useful, because the outer spin_lock_irq() disables the IRQs and flags
would always have the IRQ disabled state. Please use irqsave for the
outer call.

[snip]
> > > > > > > +
> > > > > > > +static struct v4l2_subdev *
> > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > +{
> > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > +   struct media_entity *entity;
> > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > +   struct v4l2_subdev *sensor;
> > > > > >
> > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > there no compiler warning generated for this?
> > > > > >
> > > > >
> > > > > No, there is no compiler warning.
> > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > with different compiler options.
> > > > >
> > > >
> > > > Thanks. It would be useful if you could check why the compiler you're
> > > > using doesn't show a warning here. We might be missing other
> > > > uninitialized variables.
> > > >
> > >
> > > We will feedback to your project team to check the possible reason about
> > > compiler warning issue.
> > >
> >
> > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > emerge chromeos-kernel-4_19)?
>
> > [snip]
>
> Yes, I checked this comment in the Chromium OS build environment.
> But, I think I have made the mistake here. I need to check the build
> status in the Mediatek's kernel upstream environment. I will pay
> attention in next path set upstream.
>

Thanks a lot. I will recheck this in the Chromium OS toolchain too.

> > > > > > > +
> > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > +           __func__,
> > > > > > > +           node->id,
> > > > > > > +           buf->vbb.request_fd,
> > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > +
> > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > +           return;
> > > > > >
> > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > deal with two modes.
> > > > > >
> > > > >
> > > > > The purpose of non-request function in this driver is needed by
> > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > For upstream driver, should we only consider request mode?
> > > > >
> > > >
> > > > Where does that requirement come from? Why the timing of queuing of
> > > > the buffers to the driver is important?
> > > >
> > > > [snip]
> > >
> > > Basically, this requirement comes from our internal camera
> > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > we will follow your original suggestion to keep the request mode only
> > > and remove other non-request design in other files. For upstream driver,
> > > it should support request mode only.
> > >
> >
> > Note that Chromium OS will use the "upstream driver" and we don't want
> > to diverge, so please make the userspace also use only requests. I
> > don't see a reason why there would be any need to submit any buffers
> > outside of a request.
> >
> > [snip]
>
> Ok, I have raised your concern to our colleagues and let him to discuss
> with you in another communication channel.
>

Thanks!

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-29 10:04                   ` Tomasz Figa
  (?)
@ 2019-07-30  1:44                     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-30  1:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list

On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > >                 dev_dbg(cam->dev, "jobs are full\n");
> > > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > > >                 return;
> > > >         }
> > > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > >
> > > Could we instead check the counter here and break if it's >=
> > > MTK_ISP_MAX_RUNNING_JOBS?
> > > Then we could increment it here too to simplify the code.
> > >
> >
> > Thanks for your advice.
> > We simplified this function as below:
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irq(&cam->pending_job_lock);
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> 
> Having the inner call spin_lock_irqsave() doesn't really do anything
> useful, because the outer spin_lock_irq() disables the IRQs and flags
> would always have the IRQ disabled state. Please use irqsave for the
> outer call.
> 
> [snip]

Thanks for your comment.
This is a bug which triggers one kernel warning about wrong ISR state as
you said. We have fixed it.

> > > > > > > > +
> > > > > > > > +static struct v4l2_subdev *
> > > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > > +{
> > > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > > +   struct media_entity *entity;
> > > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > > +   struct v4l2_subdev *sensor;
> > > > > > >
> > > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > > there no compiler warning generated for this?
> > > > > > >
> > > > > >
> > > > > > No, there is no compiler warning.
> > > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > > with different compiler options.
> > > > > >
> > > > >
> > > > > Thanks. It would be useful if you could check why the compiler you're
> > > > > using doesn't show a warning here. We might be missing other
> > > > > uninitialized variables.
> > > > >
> > > >
> > > > We will feedback to your project team to check the possible reason about
> > > > compiler warning issue.
> > > >
> > >
> > > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > > emerge chromeos-kernel-4_19)?
> >
> > > [snip]
> >
> > Yes, I checked this comment in the Chromium OS build environment.
> > But, I think I have made the mistake here. I need to check the build
> > status in the Mediatek's kernel upstream environment. I will pay
> > attention in next path set upstream.
> >
> 
> Thanks a lot. I will recheck this in the Chromium OS toolchain too.
> 

For these complier warnings, we have fixed them in Mediatek upstream
environment[1]. In this build environment, we could observe some
comelier warnings which are not generated by Chromium OS toolchain.

[1]
toolchain/aarch64/usr/bin/aarch64-poky-linux

> > > > > > > > +
> > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > +           __func__,
> > > > > > > > +           node->id,
> > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > +
> > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > +           return;
> > > > > > >
> > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > deal with two modes.
> > > > > > >
> > > > > >
> > > > > > The purpose of non-request function in this driver is needed by
> > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > For upstream driver, should we only consider request mode?
> > > > > >
> > > > >
> > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > the buffers to the driver is important?
> > > > >
> > > > > [snip]
> > > >
> > > > Basically, this requirement comes from our internal camera
> > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > we will follow your original suggestion to keep the request mode only
> > > > and remove other non-request design in other files. For upstream driver,
> > > > it should support request mode only.
> > > >
> > >
> > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > to diverge, so please make the userspace also use only requests. I
> > > don't see a reason why there would be any need to submit any buffers
> > > outside of a request.
> > >
> > > [snip]
> >
> > Ok, I have raised your concern to our colleagues and let him to discuss
> > with you in another communication channel.
> >
> 
> Thanks!
> 
> Best regards,
> Tomasz

Our colleague is preparing material to explain the our 3A/MW design. If
he is ready, he will discuss this with you.

In the original plan, we will deliver P1 v4 patch set tomorrow (31th
Jul.). But, there are some comments waiting for other experts' input.
Do you suggest it is better to resolve all comments before v4 patch set
submitting or continue to discuss these comments on v4?

Thanks,


Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-30  1:44                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-30  1:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: Hans Verkuil, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > >                 dev_dbg(cam->dev, "jobs are full\n");
> > > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > > >                 return;
> > > >         }
> > > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > >
> > > Could we instead check the counter here and break if it's >=
> > > MTK_ISP_MAX_RUNNING_JOBS?
> > > Then we could increment it here too to simplify the code.
> > >
> >
> > Thanks for your advice.
> > We simplified this function as below:
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irq(&cam->pending_job_lock);
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> 
> Having the inner call spin_lock_irqsave() doesn't really do anything
> useful, because the outer spin_lock_irq() disables the IRQs and flags
> would always have the IRQ disabled state. Please use irqsave for the
> outer call.
> 
> [snip]

Thanks for your comment.
This is a bug which triggers one kernel warning about wrong ISR state as
you said. We have fixed it.

> > > > > > > > +
> > > > > > > > +static struct v4l2_subdev *
> > > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > > +{
> > > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > > +   struct media_entity *entity;
> > > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > > +   struct v4l2_subdev *sensor;
> > > > > > >
> > > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > > there no compiler warning generated for this?
> > > > > > >
> > > > > >
> > > > > > No, there is no compiler warning.
> > > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > > with different compiler options.
> > > > > >
> > > > >
> > > > > Thanks. It would be useful if you could check why the compiler you're
> > > > > using doesn't show a warning here. We might be missing other
> > > > > uninitialized variables.
> > > > >
> > > >
> > > > We will feedback to your project team to check the possible reason about
> > > > compiler warning issue.
> > > >
> > >
> > > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > > emerge chromeos-kernel-4_19)?
> >
> > > [snip]
> >
> > Yes, I checked this comment in the Chromium OS build environment.
> > But, I think I have made the mistake here. I need to check the build
> > status in the Mediatek's kernel upstream environment. I will pay
> > attention in next path set upstream.
> >
> 
> Thanks a lot. I will recheck this in the Chromium OS toolchain too.
> 

For these complier warnings, we have fixed them in Mediatek upstream
environment[1]. In this build environment, we could observe some
comelier warnings which are not generated by Chromium OS toolchain.

[1]
toolchain/aarch64/usr/bin/aarch64-poky-linux

> > > > > > > > +
> > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > +           __func__,
> > > > > > > > +           node->id,
> > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > +
> > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > +           return;
> > > > > > >
> > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > deal with two modes.
> > > > > > >
> > > > > >
> > > > > > The purpose of non-request function in this driver is needed by
> > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > For upstream driver, should we only consider request mode?
> > > > > >
> > > > >
> > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > the buffers to the driver is important?
> > > > >
> > > > > [snip]
> > > >
> > > > Basically, this requirement comes from our internal camera
> > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > we will follow your original suggestion to keep the request mode only
> > > > and remove other non-request design in other files. For upstream driver,
> > > > it should support request mode only.
> > > >
> > >
> > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > to diverge, so please make the userspace also use only requests. I
> > > don't see a reason why there would be any need to submit any buffers
> > > outside of a request.
> > >
> > > [snip]
> >
> > Ok, I have raised your concern to our colleagues and let him to discuss
> > with you in another communication channel.
> >
> 
> Thanks!
> 
> Best regards,
> Tomasz

Our colleague is preparing material to explain the our 3A/MW design. If
he is ready, he will discuss this with you.

In the original plan, we will deliver P1 v4 patch set tomorrow (31th
Jul.). But, there are some comments waiting for other experts' input.
Do you suggest it is better to resolve all comments before v4 patch set
submitting or continue to discuss these comments on v4?

Thanks,


Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-07-30  1:44                     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-07-30  1:44 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
> [snip]
> > > >                 dev_dbg(cam->dev, "jobs are full\n");
> > > >                 spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > > >                 return;
> > > >         }
> > > >         list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > >
> > > Could we instead check the counter here and break if it's >=
> > > MTK_ISP_MAX_RUNNING_JOBS?
> > > Then we could increment it here too to simplify the code.
> > >
> >
> > Thanks for your advice.
> > We simplified this function as below:
> >
> > void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > {
> >         struct mtk_cam_dev_request *req, *req_prev;
> >         unsigned long flags;
> >
> >         if (!cam->streaming) {
> >                 dev_dbg(cam->dev, "stream is off\n");
> >                 return;
> >         }
> >
> >         spin_lock_irq(&cam->pending_job_lock);
> >         spin_lock_irqsave(&cam->running_job_lock, flags);
> 
> Having the inner call spin_lock_irqsave() doesn't really do anything
> useful, because the outer spin_lock_irq() disables the IRQs and flags
> would always have the IRQ disabled state. Please use irqsave for the
> outer call.
> 
> [snip]

Thanks for your comment.
This is a bug which triggers one kernel warning about wrong ISR state as
you said. We have fixed it.

> > > > > > > > +
> > > > > > > > +static struct v4l2_subdev *
> > > > > > > > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam_dev)
> > > > > > > > +{
> > > > > > > > +   struct media_device *mdev = cam_dev->seninf->entity.graph_obj.mdev;
> > > > > > > > +   struct media_entity *entity;
> > > > > > > > +   struct device *dev = &cam_dev->pdev->dev;
> > > > > > > > +   struct v4l2_subdev *sensor;
> > > > > > >
> > > > > > > This variable would be unitialized if there is no streaming sensor. Was
> > > > > > > there no compiler warning generated for this?
> > > > > > >
> > > > > >
> > > > > > No, there is no compiler warning.
> > > > > > But, we will assign sensor to NULL to avoid unnecessary compiler warning
> > > > > > with different compiler options.
> > > > > >
> > > > >
> > > > > Thanks. It would be useful if you could check why the compiler you're
> > > > > using doesn't show a warning here. We might be missing other
> > > > > uninitialized variables.
> > > > >
> > > >
> > > > We will feedback to your project team to check the possible reason about
> > > > compiler warning issue.
> > > >
> > >
> > > Do you mean that it was the Clang toolchain used on Chromium OS (e.g.
> > > emerge chromeos-kernel-4_19)?
> >
> > > [snip]
> >
> > Yes, I checked this comment in the Chromium OS build environment.
> > But, I think I have made the mistake here. I need to check the build
> > status in the Mediatek's kernel upstream environment. I will pay
> > attention in next path set upstream.
> >
> 
> Thanks a lot. I will recheck this in the Chromium OS toolchain too.
> 

For these complier warnings, we have fixed them in Mediatek upstream
environment[1]. In this build environment, we could observe some
comelier warnings which are not generated by Chromium OS toolchain.

[1]
toolchain/aarch64/usr/bin/aarch64-poky-linux

> > > > > > > > +
> > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > +           __func__,
> > > > > > > > +           node->id,
> > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > +
> > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > +           return;
> > > > > > >
> > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > deal with two modes.
> > > > > > >
> > > > > >
> > > > > > The purpose of non-request function in this driver is needed by
> > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > For upstream driver, should we only consider request mode?
> > > > > >
> > > > >
> > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > the buffers to the driver is important?
> > > > >
> > > > > [snip]
> > > >
> > > > Basically, this requirement comes from our internal camera
> > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > we will follow your original suggestion to keep the request mode only
> > > > and remove other non-request design in other files. For upstream driver,
> > > > it should support request mode only.
> > > >
> > >
> > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > to diverge, so please make the userspace also use only requests. I
> > > don't see a reason why there would be any need to submit any buffers
> > > outside of a request.
> > >
> > > [snip]
> >
> > Ok, I have raised your concern to our colleagues and let him to discuss
> > with you in another communication channel.
> >
> 
> Thanks!
> 
> Best regards,
> Tomasz

Our colleague is preparing material to explain the our 3A/MW design. If
he is ready, he will discuss this with you.

In the original plan, we will deliver P1 v4 patch set tomorrow (31th
Jul.). But, there are some comments waiting for other experts' input.
Do you suggest it is better to resolve all comments before v4 patch set
submitting or continue to discuss these comments on v4?

Thanks,


Jungo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
  2019-07-30  1:44                     ` Jungo Lin
  (?)
@ 2019-08-05  9:59                       ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-05  9:59 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree-u79uwXL29TY76Z2rM5mHXA,
	Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw,
	Frederic Chen (陳俊元)

Hi Jungo,

On Tue, Jul 30, 2019 at 10:45 AM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
>
> On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> > On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org> wrote:
> > > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > > > > > > > +
> > > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > > +           __func__,
> > > > > > > > > +           node->id,
> > > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > > +
> > > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > > +           return;
> > > > > > > >
> > > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > > deal with two modes.
> > > > > > > >
> > > > > > >
> > > > > > > The purpose of non-request function in this driver is needed by
> > > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > > For upstream driver, should we only consider request mode?
> > > > > > >
> > > > > >
> > > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > > the buffers to the driver is important?
> > > > > >
> > > > > > [snip]
> > > > >
> > > > > Basically, this requirement comes from our internal camera
> > > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > > we will follow your original suggestion to keep the request mode only
> > > > > and remove other non-request design in other files. For upstream driver,
> > > > > it should support request mode only.
> > > > >
> > > >
> > > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > > to diverge, so please make the userspace also use only requests. I
> > > > don't see a reason why there would be any need to submit any buffers
> > > > outside of a request.
> > > >
> > > > [snip]
> > >
> > > Ok, I have raised your concern to our colleagues and let him to discuss
> > > with you in another communication channel.
> > >
> >
> > Thanks!
> >
> > Best regards,
> > Tomasz
>
> Our colleague is preparing material to explain the our 3A/MW design. If
> he is ready, he will discuss this with you.

Thanks!

>
> In the original plan, we will deliver P1 v4 patch set tomorrow (31th
> Jul.). But, there are some comments waiting for other experts' input.
> Do you suggest it is better to resolve all comments before v4 patch set
> submitting or continue to discuss these comments on v4?

For the remaining v4l2-compliance issues, we can postpone them and
keep on a TODO list in the next version.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-08-05  9:59                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-05  9:59 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	ddavenport, Sj Huang, moderated list:ARM/Mediatek SoC support,
	Laurent Pinchart, Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi Jungo,

On Tue, Jul 30, 2019 at 10:45 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> > On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > > > > > > > +
> > > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > > +           __func__,
> > > > > > > > > +           node->id,
> > > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > > +
> > > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > > +           return;
> > > > > > > >
> > > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > > deal with two modes.
> > > > > > > >
> > > > > > >
> > > > > > > The purpose of non-request function in this driver is needed by
> > > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > > For upstream driver, should we only consider request mode?
> > > > > > >
> > > > > >
> > > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > > the buffers to the driver is important?
> > > > > >
> > > > > > [snip]
> > > > >
> > > > > Basically, this requirement comes from our internal camera
> > > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > > we will follow your original suggestion to keep the request mode only
> > > > > and remove other non-request design in other files. For upstream driver,
> > > > > it should support request mode only.
> > > > >
> > > >
> > > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > > to diverge, so please make the userspace also use only requests. I
> > > > don't see a reason why there would be any need to submit any buffers
> > > > outside of a request.
> > > >
> > > > [snip]
> > >
> > > Ok, I have raised your concern to our colleagues and let him to discuss
> > > with you in another communication channel.
> > >
> >
> > Thanks!
> >
> > Best regards,
> > Tomasz
>
> Our colleague is preparing material to explain the our 3A/MW design. If
> he is ready, he will discuss this with you.

Thanks!

>
> In the original plan, we will deliver P1 v4 patch set tomorrow (31th
> Jul.). But, there are some comments waiting for other experts' input.
> Do you suggest it is better to resolve all comments before v4 patch set
> submitting or continue to discuss these comments on v4?

For the remaining v4l2-compliance issues, we can postpone them and
keep on a TODO list in the next version.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions
@ 2019-08-05  9:59                       ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-05  9:59 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Tue, Jul 30, 2019 at 10:45 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> On Mon, 2019-07-29 at 19:04 +0900, Tomasz Figa wrote:
> > On Mon, Jul 29, 2019 at 10:18 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > On Fri, 2019-07-26 at 14:49 +0900, Tomasz Figa wrote:
> > > > On Wed, Jul 24, 2019 at 1:31 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > On Tue, 2019-07-23 at 19:21 +0900, Tomasz Figa wrote:
> > > > > > On Thu, Jul 18, 2019 at 1:39 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > > > > On Wed, 2019-07-10 at 18:54 +0900, Tomasz Figa wrote:
> > > > > > > > On Tue, Jun 11, 2019 at 11:53:41AM +0800, Jungo Lin wrote:
[snip]
> > > > > > > > > +
> > > > > > > > > +   dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n",
> > > > > > > > > +           __func__,
> > > > > > > > > +           node->id,
> > > > > > > > > +           buf->vbb.request_fd,
> > > > > > > > > +           buf->vbb.vb2_buf.index);
> > > > > > > > > +
> > > > > > > > > +   /* For request buffers en-queue, handled in mtk_cam_req_try_queue */
> > > > > > > > > +   if (vb->vb2_queue->uses_requests)
> > > > > > > > > +           return;
> > > > > > > >
> > > > > > > > I'd suggest removing non-request support from this driver. Even if we end up
> > > > > > > > with a need to provide compatibility for non-request mode, then it should be
> > > > > > > > built on top of the requests mode, so that the driver itself doesn't have to
> > > > > > > > deal with two modes.
> > > > > > > >
> > > > > > >
> > > > > > > The purpose of non-request function in this driver is needed by
> > > > > > > our camera middle-ware design. It needs 3A statistics buffers before
> > > > > > > image buffers en-queue. So we need to en-queue 3A statistics with
> > > > > > > non-request mode in this driver. After MW got the 3A statistics data, it
> > > > > > > will en-queue the images, tuning buffer and other meta buffers with
> > > > > > > request mode. Based on this requirement, do you have any suggestion?
> > > > > > > For upstream driver, should we only consider request mode?
> > > > > > >
> > > > > >
> > > > > > Where does that requirement come from? Why the timing of queuing of
> > > > > > the buffers to the driver is important?
> > > > > >
> > > > > > [snip]
> > > > >
> > > > > Basically, this requirement comes from our internal camera
> > > > > middle-ware/3A hal in user space. Since this is not generic requirement,
> > > > > we will follow your original suggestion to keep the request mode only
> > > > > and remove other non-request design in other files. For upstream driver,
> > > > > it should support request mode only.
> > > > >
> > > >
> > > > Note that Chromium OS will use the "upstream driver" and we don't want
> > > > to diverge, so please make the userspace also use only requests. I
> > > > don't see a reason why there would be any need to submit any buffers
> > > > outside of a request.
> > > >
> > > > [snip]
> > >
> > > Ok, I have raised your concern to our colleagues and let him to discuss
> > > with you in another communication channel.
> > >
> >
> > Thanks!
> >
> > Best regards,
> > Tomasz
>
> Our colleague is preparing material to explain the our 3A/MW design. If
> he is ready, he will discuss this with you.

Thanks!

>
> In the original plan, we will deliver P1 v4 patch set tomorrow (31th
> Jul.). But, there are some comments waiting for other experts' input.
> Do you suggest it is better to resolve all comments before v4 patch set
> submitting or continue to discuss these comments on v4?

For the remaining v4l2-compliance issues, we can postpone them and
keep on a TODO list in the next version.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-07-26  7:23             ` Jungo Lin
  (?)
@ 2019-08-06  9:47               ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-06  9:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab

Hi Jungo,

On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > .Hi Jungo,
> >
> > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
[snip]
> > > > > +
> > > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > > +
> > > > > +   /* Sof, done order check */
> > > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > > +
> > > > > +           /* Notify IRQ event and enqueue frame */
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > > +           isp_dev->current_frame = hw_frame_num;
> > > >
> > > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > > event?
> > > >
> > >
> > > This is a another spare register for frame sequence number usage.
> > > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > > Is it a better understanding?
> >
> > I'm sorry, unfortunately it's still not clear to me. Is it the
> > sequence number of the frame that was just processed and returned to
> > the kernel or the next frame that is going to be processed from now
> > on?
> >
>
> It is the next frame that is going to be proceed.
> We simplify the implementation of isp_irq_cam function. The hw_frame_num
> is renamed to dequeue_frame_seq_no and saved this value from HW at
> SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
> next frame to be processed. If there is SW_PASS1_DON_ST, it means this
> frame is processed done. We use this value to de-queue the frame request
> and return buffers to VB2.
>
> The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
> HW_PASS1_DON_ST.
>
> a. SW_PASS_DON_ST is designed for DMAs done event.
> If there is no available DMA buffers en-queued into HW, there is no
> SW_PADD_DON_ST.
>
> b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
> It is paired with SOF IRQ event, even if there is no available DMA
> buffers.
>
> static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>                                unsigned int dequeue_frame_seq_no)
> {
>         dma_addr_t base_addr = p1_dev->composer_iova;
>         int composed_frame_seq_no =
> atomic_read(&p1_dev->composed_frame_seq_no);
>         unsigned int addr_offset;
>
>         /* Send V4L2_EVENT_FRAME_SYNC event */
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
>
>         p1_dev->sof_count += 1;
>         /* Save dequeue frame information */
>         p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;
>
>         /* Update CQ base address if needed */
>         if (composed_frame_seq_no <= dequeue_frame_seq_no) {
>                 dev_dbg(p1_dev->dev,
>                         "SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
>                         composed_frame_seq_no, dequeue_frame_seq_no);
>                 return;
>         }
>         addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>                 (dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>         writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>         dev_dbg(p1_dev->dev,
>                 "SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
>                 composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
> }
>
> void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>                                    unsigned int frame_seq_no)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>                 dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>                         req->frame_params.frame_seq_no, frame_seq_no);
>
>                 /* Match by the en-queued request number */
>                 if (req->frame_params.frame_seq_no == frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>                         list_del(&req->list);
>                         break;
>                 } else if (req->frame_params.frame_seq_no < frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space for frame drop */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>                         dev_warn(cam->dev, "frame_seq:%d drop\n",
>                                  req->frame_params.frame_seq_no);
>                         list_del(&req->list);
>                 } else {
>                         break;
>                 }
>         }
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
>
> static irqreturn_t isp_irq_cam(int irq, void *data)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
>         struct device *dev = p1_dev->dev;
>         unsigned int dequeue_frame_seq_no;
>         unsigned int irq_status, err_status, dma_status;
>         unsigned long flags;
>
>         spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
>         irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
>         err_status = irq_status & INT_ST_MASK_CAM_ERR;
>         dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
>         dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
>         spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
>
>         /*
>          * In normal case, the next SOF ISR should come after HW PASS1 DONE
> ISR.
>          * If these two ISRs come together, print warning msg to hint.
>          */
>         if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>                 dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
>         /* De-queue frame */
>         if (irq_status & SW_PASS1_DON_ST) {
>                 mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
>                                               dequeue_frame_seq_no);
>                 mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
>         }
>
>         /* Save frame info. & update CQ address for frame HW en-queue */
>         if (irq_status & SOF_INT_ST)
>                 isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);
>
>         /* Check ISP error status */
>         if (err_status) {
>                 dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
>                 /* Show DMA errors in detail */
>                 if (err_status & DMA_ERR_ST)
>                         isp_irq_handle_dma_err(p1_dev);
>         }
>
>         dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
>                 p1_dev->sof_count, irq_status, dma_status,
>                 dequeue_frame_seq_no);
>
>         return IRQ_HANDLED;
> }

I think I understand this now and the code above also looks good to
me. Thanks a lot!

>
> > >
> > > Below is our frame request handling in current design.
> > >
> > > 1. Buffer preparation
> > > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > > other meta histogram buffers (LCSO/LMVO) into one request.
> > > - Accumulated one unique frame sequence number to each request and send
> > > this request to the SCP composer to compose CQ (Command queue) buffer
> > > via SCP_ISP_FRAME IPI command.
> > > - CQ buffer is frame registers set. If ISP registers should be updated
> > > per frame, these registers are configured in the CQ buffer, such as
> > > frame sequence number, DMA addresses and tuning ISP registers.
> > > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > > its corresponding frame sequence number. Based on this, kernel driver
> > > knows which request is ready to be en-queued and save this with
> > > p1_dev->isp_ctx.composed_frame_id.
> >
> > Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> > already have a linked lists of requests that are composed and ready to
> > be enqueued? Also, the request itself would contain its frame ID
> > inside the driver request struct, right?
> >
>
> Below is current implementation for frame request en-queued.
> Before en-queued into HW by CQ, the request should be composed by SCP
> composer.
>
> a. mtk_cam_dev_req_try_queue()
> - Insert the request into p1_dev->running_job_list
> b. mtk_isp_req_enqueue()
> - Assign new next frame ID to this request.
> - Sending to SCP by workqueue
> - This request is ready to compose
> c. isp_tx_frame_worker()
> - Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
> - SCP composer will compose the buffer CQ for this request frame based
> on struct mtk_p1_frame_param which includes frame ID.
> - If scp_ipi_send() is returned, it means the request is composed done.
> Or
> d. isp_composer_handler()
> - If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
> the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
> - The request is composed done here.
> e. isp_irq_handle_sof()
> - In SOF timing, we will check there is any available composed CQ
> buffers by comparing composed & current de-queued frame ID.
>
> For p1_dev->running_job_list, we can't guarantee the requests are
> composed until the end of step c. For step e, we need to know how many
> available composed requests are ready to en-queued.
>
> Do you suggest we add another new link-list to save these requests in
> step c or we could update p1_dev->composed_frame_seq_no in step c and
> remove the implementation in step d[1]?

Okay, thanks to your explanation above I think I understood how the
hardware flow behaves and so I think we can indeed keep the
composed_frame_seq counter. Thanks!

>
> [1]. isp_composer_handler() is mandatory callback function for SCP
> sending API with sync mode design.
>
> static void isp_composer_handler(void *data, unsigned int len, void
> *priv)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
>         struct mtk_isp_scp_p1_cmd *ipi_msg;
>
>         ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
>
>         if (ipi_msg->cmd_id != ISP_CMD_ACK)
>                 return;
>
>         if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
>                 atomic_set(&p1_dev->composed_frame_seq_no,
>                            ipi_msg->ack_info.frame_seq_no);
>                 dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
>                         p1_dev->composed_frame_seq_no);
>         }
> }
>
> > > - The maximum number of CQ buffers in SCP is 3.
> > >
> > > 2. Buffer en-queue flow
> > > - In order to configure correct CQ buffer setting before next SQF event,
> > > it is depended on by MTK ISP P1 HW CQ mechanism.
> > > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > > when HW_PASS1_DON_ST is received which means DMA output is done.
> > > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > > bypass.
> > > - So we will check available CQ buffers by comparing composed frame
> > > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > > - If there are available CQ buffers, update the CQ base address to the
> > > next CQ buffer address based on current de-enqueue frame sequence
> > > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > > request until request is done.
> > > - But, for the first request, it is loaded into HW manually when
> > > streaming is on for better performance.
> > >
> > > 3. Buffer de-queue flow
> > > - We will use frame sequence number to decide which request is ready to
> > > de-queue.
> > > - We will save some important register setting from ISP HW when SOF is
> > > received. This is because the ISP HW starts to output the data with the
> > > corresponding settings, especially frame sequence number setting.
> >
> > Could you explain a bit more about these important register settings?
> > When does the hardware update the values in the register to new ones?
> > At SOF?
> >
>
> Sorry about my words.
> In the current implementation, we just save frame ID.
>

Ah, okay, makes sense. No worries. :)

>
> > > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > > done. So we could call isp_deque_request_frame with frame sequence
> > > number to de-queue frame to VB2
> >
> > What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> >
>
> This is explained above.
>
> > > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > > AAO/AFO non-request buffers are en-queued without request buffers at the
> > > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> >
> > Do we have a case like this? Wouldn't we normally always want to
> > bundle AAO/AFO buffers with frame buffers?
> >
>
> For upstream driver, we will remove non-request design.
>

I think we also talked about a thing related to this in the thread for
another patch from this series. Basically on Chrome OS we want to use
the upstream driver, so corresponding userspace changes might be
needed as well.

> > > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > > we may try to send another request to SCP for composing.
> >
> > Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> > buffers, right?
> >
>
> We expected the the life cycle of CQ buffer is same as frame request.
> So SW_PASS1_DON_ST is good timing to re-queue the next request to
> compose.
> For the CQ operations, we will explain later.
>
> > >
> > > Hopefully, my explanation is helpful for better understanding our
> > > implementation. If you still have any questions, please let me know.
> > >
> >
> > Yes, it's more clear now, thanks. Still some more comments above, though.
> >
> > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +   } else {
> > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +           }
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > +   }
> > > >
> > > > The if and else blocks do almost the same things just in different order. Is
> > > > it really expected?
> > > >
> > >
> > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > any de-queued frame and update the next frame setting later.
> > > Normally, this is a corner case or system performance issue.
> >
> > So it sounds like HW_PASS1_DON_ST means that all data from current
> > frame has been written, right? If I understand your explanation above
> > correctly, that would mean following handling of each interrupt:
> >
> > HW_PASS1_DON_ST:
> >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > is this handled? does the CQ hardware automatically receive this event
> > from the ISP hadware?
> >  - return VB2 buffers,
> >  - complete requests.
> >
> > SOF_INT_ST:
> >  - send VSYNC event to userspace,
> >  - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> >  - reclaim CQ buffer and enqueue next frame to composing if available
> >
>
> Sorry for our implementation of HW_PASS1_DON_ST.
> It is confusing.
> Below is the revised version based on your conclusion.
> So in our new implemmenation, we just handle SOF_INT_ST &
> SW_PASS1_DON_ST events. We just add one warning message for
> HW_PASS1_DON_ST
>
> HW_PASS1_DON_ST:
> - CQ executes with next CQ buffer to prepare for next frame.
>
> SOF_INT_ST:
> - send VSYNC event to userspace,
> - program next CQ buffer to CQ,
>
> SW_PASS1_DON_ST:
> - reclaim CQ buffer and enqueue next frame to composing if available
> - return VB2 buffers,
> - complete requests.
>
> For CQ HW operations, it is listed below:
>
> a. The CQ buffer has two kinds of information
>  - Which ISP registers needs to be updated.
>  - Where the corresponding ISP register data to be read.
> b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> event periodically.
>  - Normally, if the ISP HW receives the completed frame and it will
> trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> -  So the CQ buffer loading is performed by ISP HW automatically.
> c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> to decide which CQ buffer is loaded.
>    - So we configure the next CQ base address in SOF.
> d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> and update the ISP register values into HW.
>    - SCP composer will compose one dummy CQ buffer and assign it to
> REG_CQ_THR0_BASEADDR of each CQ buffer.
>    - Dummy CQ buffer has no updated ISP registers comparing with other
> CQ buffers.
>    - With this design, if there is no updated new CQ buffer by driver
> which may be caused no en-queue frames from user space. The CQ HW will
> load dummy CQ buffer and do nothing.

Does the set of registers programmed by CQ include destination buffer
addresses to? If yes, we would end up overwriting previous frames if
no new buffers are provided.

> f. The CQ buffer loading is guaranteed by HW to finish before the next
> SOF.
>

Okay, thanks a lot for the explanation. This is much more clear now.

[snip]
> > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > >
> > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > or is about to become active. System PM ops would be called at system power
> > > > state change and the hardware might be both idle or active. Please also see
> > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > >
> > >
> > > Here is the new implementation. It should be clear to show the
> > > difference between system and runtime PM ops.
> > >
> > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > >                                 pm_runtime_force_resume)
> > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > NULL)
> > > };
> >
> > That's still not correct. In runtime suspend/resume ops we already are
> > not streaming anymore, because we call pm_runtime_get/put_*() when
> > starting and stopping streaming. In system suspend/resume ops we might
> > be streaming and that's when we need to stop the hardware and wait for
> > it to finish. Please implement these ops separately.
> >
> > Best regards,
> > Tomasz
>
>
> Ok, got your point.
> Below is the new implementation for your review.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Check ISP is streaming or not */
>         if (!p1_dev->cam_dev.streaming)
>                 goto done;

We would normally check here for pm_runtime_suspended(). Although they
both should be equivalent. Still, there is no need to call
pm_runtime_force_suspend() if the latter is true, so we could just
return 0 instantly.

>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "Cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
> done:
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;
>
>         return 0;
> }
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Force ISP HW to resume if needed */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;

We should do this conditionally based on what pm_runtime_suspended()
returns. If it's non-zero then we can just return 0 instantly.

>
>         if (!p1_dev->cam_dev.streaming)
>                 return 0;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

Sorry, I guess I wasn't clear in my reply. It's not possible to get
this callback called if the device is already runtime suspended.

>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

In this case the above call would always return non-zero, so the
behavior wouldn't be very good.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-06  9:47               ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-06  9:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: Hans Verkuil, Laurent Pinchart, Matthias Brugger,
	Mauro Carvalho Chehab, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	devicetree, srv_heupstream, ddavenport, Rob Herring,
	Sean Cheng (鄭昇弘),
	Sj Huang, Frederic Chen (陳俊元),
	Ryan Yu (余孟修),
	Rynn Wu (吳育恩),
	Frankie Chiu (邱文凱)

Hi Jungo,

On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > .Hi Jungo,
> >
> > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
[snip]
> > > > > +
> > > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > > +
> > > > > +   /* Sof, done order check */
> > > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > > +
> > > > > +           /* Notify IRQ event and enqueue frame */
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > > +           isp_dev->current_frame = hw_frame_num;
> > > >
> > > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > > event?
> > > >
> > >
> > > This is a another spare register for frame sequence number usage.
> > > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > > Is it a better understanding?
> >
> > I'm sorry, unfortunately it's still not clear to me. Is it the
> > sequence number of the frame that was just processed and returned to
> > the kernel or the next frame that is going to be processed from now
> > on?
> >
>
> It is the next frame that is going to be proceed.
> We simplify the implementation of isp_irq_cam function. The hw_frame_num
> is renamed to dequeue_frame_seq_no and saved this value from HW at
> SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
> next frame to be processed. If there is SW_PASS1_DON_ST, it means this
> frame is processed done. We use this value to de-queue the frame request
> and return buffers to VB2.
>
> The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
> HW_PASS1_DON_ST.
>
> a. SW_PASS_DON_ST is designed for DMAs done event.
> If there is no available DMA buffers en-queued into HW, there is no
> SW_PADD_DON_ST.
>
> b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
> It is paired with SOF IRQ event, even if there is no available DMA
> buffers.
>
> static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>                                unsigned int dequeue_frame_seq_no)
> {
>         dma_addr_t base_addr = p1_dev->composer_iova;
>         int composed_frame_seq_no =
> atomic_read(&p1_dev->composed_frame_seq_no);
>         unsigned int addr_offset;
>
>         /* Send V4L2_EVENT_FRAME_SYNC event */
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
>
>         p1_dev->sof_count += 1;
>         /* Save dequeue frame information */
>         p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;
>
>         /* Update CQ base address if needed */
>         if (composed_frame_seq_no <= dequeue_frame_seq_no) {
>                 dev_dbg(p1_dev->dev,
>                         "SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
>                         composed_frame_seq_no, dequeue_frame_seq_no);
>                 return;
>         }
>         addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>                 (dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>         writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>         dev_dbg(p1_dev->dev,
>                 "SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
>                 composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
> }
>
> void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>                                    unsigned int frame_seq_no)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>                 dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>                         req->frame_params.frame_seq_no, frame_seq_no);
>
>                 /* Match by the en-queued request number */
>                 if (req->frame_params.frame_seq_no == frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>                         list_del(&req->list);
>                         break;
>                 } else if (req->frame_params.frame_seq_no < frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space for frame drop */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>                         dev_warn(cam->dev, "frame_seq:%d drop\n",
>                                  req->frame_params.frame_seq_no);
>                         list_del(&req->list);
>                 } else {
>                         break;
>                 }
>         }
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
>
> static irqreturn_t isp_irq_cam(int irq, void *data)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
>         struct device *dev = p1_dev->dev;
>         unsigned int dequeue_frame_seq_no;
>         unsigned int irq_status, err_status, dma_status;
>         unsigned long flags;
>
>         spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
>         irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
>         err_status = irq_status & INT_ST_MASK_CAM_ERR;
>         dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
>         dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
>         spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
>
>         /*
>          * In normal case, the next SOF ISR should come after HW PASS1 DONE
> ISR.
>          * If these two ISRs come together, print warning msg to hint.
>          */
>         if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>                 dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
>         /* De-queue frame */
>         if (irq_status & SW_PASS1_DON_ST) {
>                 mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
>                                               dequeue_frame_seq_no);
>                 mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
>         }
>
>         /* Save frame info. & update CQ address for frame HW en-queue */
>         if (irq_status & SOF_INT_ST)
>                 isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);
>
>         /* Check ISP error status */
>         if (err_status) {
>                 dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
>                 /* Show DMA errors in detail */
>                 if (err_status & DMA_ERR_ST)
>                         isp_irq_handle_dma_err(p1_dev);
>         }
>
>         dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
>                 p1_dev->sof_count, irq_status, dma_status,
>                 dequeue_frame_seq_no);
>
>         return IRQ_HANDLED;
> }

I think I understand this now and the code above also looks good to
me. Thanks a lot!

>
> > >
> > > Below is our frame request handling in current design.
> > >
> > > 1. Buffer preparation
> > > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > > other meta histogram buffers (LCSO/LMVO) into one request.
> > > - Accumulated one unique frame sequence number to each request and send
> > > this request to the SCP composer to compose CQ (Command queue) buffer
> > > via SCP_ISP_FRAME IPI command.
> > > - CQ buffer is frame registers set. If ISP registers should be updated
> > > per frame, these registers are configured in the CQ buffer, such as
> > > frame sequence number, DMA addresses and tuning ISP registers.
> > > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > > its corresponding frame sequence number. Based on this, kernel driver
> > > knows which request is ready to be en-queued and save this with
> > > p1_dev->isp_ctx.composed_frame_id.
> >
> > Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> > already have a linked lists of requests that are composed and ready to
> > be enqueued? Also, the request itself would contain its frame ID
> > inside the driver request struct, right?
> >
>
> Below is current implementation for frame request en-queued.
> Before en-queued into HW by CQ, the request should be composed by SCP
> composer.
>
> a. mtk_cam_dev_req_try_queue()
> - Insert the request into p1_dev->running_job_list
> b. mtk_isp_req_enqueue()
> - Assign new next frame ID to this request.
> - Sending to SCP by workqueue
> - This request is ready to compose
> c. isp_tx_frame_worker()
> - Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
> - SCP composer will compose the buffer CQ for this request frame based
> on struct mtk_p1_frame_param which includes frame ID.
> - If scp_ipi_send() is returned, it means the request is composed done.
> Or
> d. isp_composer_handler()
> - If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
> the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
> - The request is composed done here.
> e. isp_irq_handle_sof()
> - In SOF timing, we will check there is any available composed CQ
> buffers by comparing composed & current de-queued frame ID.
>
> For p1_dev->running_job_list, we can't guarantee the requests are
> composed until the end of step c. For step e, we need to know how many
> available composed requests are ready to en-queued.
>
> Do you suggest we add another new link-list to save these requests in
> step c or we could update p1_dev->composed_frame_seq_no in step c and
> remove the implementation in step d[1]?

Okay, thanks to your explanation above I think I understood how the
hardware flow behaves and so I think we can indeed keep the
composed_frame_seq counter. Thanks!

>
> [1]. isp_composer_handler() is mandatory callback function for SCP
> sending API with sync mode design.
>
> static void isp_composer_handler(void *data, unsigned int len, void
> *priv)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
>         struct mtk_isp_scp_p1_cmd *ipi_msg;
>
>         ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
>
>         if (ipi_msg->cmd_id != ISP_CMD_ACK)
>                 return;
>
>         if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
>                 atomic_set(&p1_dev->composed_frame_seq_no,
>                            ipi_msg->ack_info.frame_seq_no);
>                 dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
>                         p1_dev->composed_frame_seq_no);
>         }
> }
>
> > > - The maximum number of CQ buffers in SCP is 3.
> > >
> > > 2. Buffer en-queue flow
> > > - In order to configure correct CQ buffer setting before next SQF event,
> > > it is depended on by MTK ISP P1 HW CQ mechanism.
> > > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > > when HW_PASS1_DON_ST is received which means DMA output is done.
> > > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > > bypass.
> > > - So we will check available CQ buffers by comparing composed frame
> > > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > > - If there are available CQ buffers, update the CQ base address to the
> > > next CQ buffer address based on current de-enqueue frame sequence
> > > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > > request until request is done.
> > > - But, for the first request, it is loaded into HW manually when
> > > streaming is on for better performance.
> > >
> > > 3. Buffer de-queue flow
> > > - We will use frame sequence number to decide which request is ready to
> > > de-queue.
> > > - We will save some important register setting from ISP HW when SOF is
> > > received. This is because the ISP HW starts to output the data with the
> > > corresponding settings, especially frame sequence number setting.
> >
> > Could you explain a bit more about these important register settings?
> > When does the hardware update the values in the register to new ones?
> > At SOF?
> >
>
> Sorry about my words.
> In the current implementation, we just save frame ID.
>

Ah, okay, makes sense. No worries. :)

>
> > > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > > done. So we could call isp_deque_request_frame with frame sequence
> > > number to de-queue frame to VB2
> >
> > What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> >
>
> This is explained above.
>
> > > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > > AAO/AFO non-request buffers are en-queued without request buffers at the
> > > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> >
> > Do we have a case like this? Wouldn't we normally always want to
> > bundle AAO/AFO buffers with frame buffers?
> >
>
> For upstream driver, we will remove non-request design.
>

I think we also talked about a thing related to this in the thread for
another patch from this series. Basically on Chrome OS we want to use
the upstream driver, so corresponding userspace changes might be
needed as well.

> > > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > > we may try to send another request to SCP for composing.
> >
> > Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> > buffers, right?
> >
>
> We expected the the life cycle of CQ buffer is same as frame request.
> So SW_PASS1_DON_ST is good timing to re-queue the next request to
> compose.
> For the CQ operations, we will explain later.
>
> > >
> > > Hopefully, my explanation is helpful for better understanding our
> > > implementation. If you still have any questions, please let me know.
> > >
> >
> > Yes, it's more clear now, thanks. Still some more comments above, though.
> >
> > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +   } else {
> > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +           }
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > +   }
> > > >
> > > > The if and else blocks do almost the same things just in different order. Is
> > > > it really expected?
> > > >
> > >
> > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > any de-queued frame and update the next frame setting later.
> > > Normally, this is a corner case or system performance issue.
> >
> > So it sounds like HW_PASS1_DON_ST means that all data from current
> > frame has been written, right? If I understand your explanation above
> > correctly, that would mean following handling of each interrupt:
> >
> > HW_PASS1_DON_ST:
> >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > is this handled? does the CQ hardware automatically receive this event
> > from the ISP hadware?
> >  - return VB2 buffers,
> >  - complete requests.
> >
> > SOF_INT_ST:
> >  - send VSYNC event to userspace,
> >  - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> >  - reclaim CQ buffer and enqueue next frame to composing if available
> >
>
> Sorry for our implementation of HW_PASS1_DON_ST.
> It is confusing.
> Below is the revised version based on your conclusion.
> So in our new implemmenation, we just handle SOF_INT_ST &
> SW_PASS1_DON_ST events. We just add one warning message for
> HW_PASS1_DON_ST
>
> HW_PASS1_DON_ST:
> - CQ executes with next CQ buffer to prepare for next frame.
>
> SOF_INT_ST:
> - send VSYNC event to userspace,
> - program next CQ buffer to CQ,
>
> SW_PASS1_DON_ST:
> - reclaim CQ buffer and enqueue next frame to composing if available
> - return VB2 buffers,
> - complete requests.
>
> For CQ HW operations, it is listed below:
>
> a. The CQ buffer has two kinds of information
>  - Which ISP registers needs to be updated.
>  - Where the corresponding ISP register data to be read.
> b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> event periodically.
>  - Normally, if the ISP HW receives the completed frame and it will
> trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> -  So the CQ buffer loading is performed by ISP HW automatically.
> c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> to decide which CQ buffer is loaded.
>    - So we configure the next CQ base address in SOF.
> d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> and update the ISP register values into HW.
>    - SCP composer will compose one dummy CQ buffer and assign it to
> REG_CQ_THR0_BASEADDR of each CQ buffer.
>    - Dummy CQ buffer has no updated ISP registers comparing with other
> CQ buffers.
>    - With this design, if there is no updated new CQ buffer by driver
> which may be caused no en-queue frames from user space. The CQ HW will
> load dummy CQ buffer and do nothing.

Does the set of registers programmed by CQ include destination buffer
addresses to? If yes, we would end up overwriting previous frames if
no new buffers are provided.

> f. The CQ buffer loading is guaranteed by HW to finish before the next
> SOF.
>

Okay, thanks a lot for the explanation. This is much more clear now.

[snip]
> > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > >
> > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > or is about to become active. System PM ops would be called at system power
> > > > state change and the hardware might be both idle or active. Please also see
> > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > >
> > >
> > > Here is the new implementation. It should be clear to show the
> > > difference between system and runtime PM ops.
> > >
> > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > >                                 pm_runtime_force_resume)
> > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > NULL)
> > > };
> >
> > That's still not correct. In runtime suspend/resume ops we already are
> > not streaming anymore, because we call pm_runtime_get/put_*() when
> > starting and stopping streaming. In system suspend/resume ops we might
> > be streaming and that's when we need to stop the hardware and wait for
> > it to finish. Please implement these ops separately.
> >
> > Best regards,
> > Tomasz
>
>
> Ok, got your point.
> Below is the new implementation for your review.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Check ISP is streaming or not */
>         if (!p1_dev->cam_dev.streaming)
>                 goto done;

We would normally check here for pm_runtime_suspended(). Although they
both should be equivalent. Still, there is no need to call
pm_runtime_force_suspend() if the latter is true, so we could just
return 0 instantly.

>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "Cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
> done:
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;
>
>         return 0;
> }
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Force ISP HW to resume if needed */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;

We should do this conditionally based on what pm_runtime_suspended()
returns. If it's non-zero then we can just return 0 instantly.

>
>         if (!p1_dev->cam_dev.streaming)
>                 return 0;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

Sorry, I guess I wasn't clear in my reply. It's not possible to get
this callback called if the device is already runtime suspended.

>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

In this case the above call would always return non-zero, so the
behavior wouldn't be very good.

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-06  9:47               ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-06  9:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

Hi Jungo,

On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > .Hi Jungo,
> >
> > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > Hi Jungo,
> > > >
> > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
[snip]
> > > > > +
> > > > > +   err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > > > > +
> > > > > +   /* Sof, done order check */
> > > > > +   if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST)) {
> > > > > +           dev_dbg(dev, "sof_done block cnt:%d\n", isp_dev->sof_count);
> > > > > +
> > > > > +           /* Notify IRQ event and enqueue frame */
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 0);
> > > > > +           isp_dev->current_frame = hw_frame_num;
> > > >
> > > > What exactly is hw_frame_num? Shouldn't we assign it before notifying the
> > > > event?
> > > >
> > >
> > > This is a another spare register for frame sequence number usage.
> > > It comes from struct p1_frame_param:frame_seq_no which is sent by
> > > SCP_ISP_FRAME IPI command. We will rename this to dequeue_frame_seq_no.
> > > Is it a better understanding?
> >
> > I'm sorry, unfortunately it's still not clear to me. Is it the
> > sequence number of the frame that was just processed and returned to
> > the kernel or the next frame that is going to be processed from now
> > on?
> >
>
> It is the next frame that is going to be proceed.
> We simplify the implementation of isp_irq_cam function. The hw_frame_num
> is renamed to dequeue_frame_seq_no and saved this value from HW at
> SOF_INT_ST. Since it is obtained in SOF_INI_ST event, it means it is
> next frame to be processed. If there is SW_PASS1_DON_ST, it means this
> frame is processed done. We use this value to de-queue the frame request
> and return buffers to VB2.
>
> The normal IRQ sequence is SOF_INT_ST => SW_PASS1_DON_ST &
> HW_PASS1_DON_ST.
>
> a. SW_PASS_DON_ST is designed for DMAs done event.
> If there is no available DMA buffers en-queued into HW, there is no
> SW_PADD_DON_ST.
>
> b. HW_PASS_DON_ST is designed to trigger CQ buffer load procedure.
> It is paired with SOF IRQ event, even if there is no available DMA
> buffers.
>
> static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>                                unsigned int dequeue_frame_seq_no)
> {
>         dma_addr_t base_addr = p1_dev->composer_iova;
>         int composed_frame_seq_no =
> atomic_read(&p1_dev->composed_frame_seq_no);
>         unsigned int addr_offset;
>
>         /* Send V4L2_EVENT_FRAME_SYNC event */
>         mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeue_frame_seq_no);
>
>         p1_dev->sof_count += 1;
>         /* Save dequeue frame information */
>         p1_dev->dequeue_frame_seq_no = dequeue_frame_seq_no;
>
>         /* Update CQ base address if needed */
>         if (composed_frame_seq_no <= dequeue_frame_seq_no) {
>                 dev_dbg(p1_dev->dev,
>                         "SOF_INT_ST, no update, cq_num:%d, frame_seq:%d",
>                         composed_frame_seq_no, dequeue_frame_seq_no);
>                 return;
>         }
>         addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>                 (dequeue_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>         writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>         dev_dbg(p1_dev->dev,
>                 "SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x",
>                 composed_frame_seq_no, dequeue_frame_seq_no, addr_offset);
> }
>
> void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>                                    unsigned int frame_seq_no)
> {
>         struct mtk_cam_dev_request *req, *req_prev;
>         unsigned long flags;
>
>         spin_lock_irqsave(&cam->running_job_lock, flags);
>         list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>                 dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>                         req->frame_params.frame_seq_no, frame_seq_no);
>
>                 /* Match by the en-queued request number */
>                 if (req->frame_params.frame_seq_no == frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>                         list_del(&req->list);
>                         break;
>                 } else if (req->frame_params.frame_seq_no < frame_seq_no) {
>                         atomic_dec(&cam->running_job_count);
>                         /* Pass to user space for frame drop */
>                         mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>                         dev_warn(cam->dev, "frame_seq:%d drop\n",
>                                  req->frame_params.frame_seq_no);
>                         list_del(&req->list);
>                 } else {
>                         break;
>                 }
>         }
>         spin_unlock_irqrestore(&cam->running_job_lock, flags);
>
> static irqreturn_t isp_irq_cam(int irq, void *data)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
>         struct device *dev = p1_dev->dev;
>         unsigned int dequeue_frame_seq_no;
>         unsigned int irq_status, err_status, dma_status;
>         unsigned long flags;
>
>         spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
>         irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
>         err_status = irq_status & INT_ST_MASK_CAM_ERR;
>         dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
>         dequeue_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
>         spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
>
>         /*
>          * In normal case, the next SOF ISR should come after HW PASS1 DONE
> ISR.
>          * If these two ISRs come together, print warning msg to hint.
>          */
>         if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>                 dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>
>         /* De-queue frame */
>         if (irq_status & SW_PASS1_DON_ST) {
>                 mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
>                                               dequeue_frame_seq_no);
>                 mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
>         }
>
>         /* Save frame info. & update CQ address for frame HW en-queue */
>         if (irq_status & SOF_INT_ST)
>                 isp_irq_handle_sof(p1_dev, dequeue_frame_seq_no);
>
>         /* Check ISP error status */
>         if (err_status) {
>                 dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
>                 /* Show DMA errors in detail */
>                 if (err_status & DMA_ERR_ST)
>                         isp_irq_handle_dma_err(p1_dev);
>         }
>
>         dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d",
>                 p1_dev->sof_count, irq_status, dma_status,
>                 dequeue_frame_seq_no);
>
>         return IRQ_HANDLED;
> }

I think I understand this now and the code above also looks good to
me. Thanks a lot!

>
> > >
> > > Below is our frame request handling in current design.
> > >
> > > 1. Buffer preparation
> > > - Combined image buffers (IMGO/RRZO) + meta input buffer (Tuining) +
> > > other meta histogram buffers (LCSO/LMVO) into one request.
> > > - Accumulated one unique frame sequence number to each request and send
> > > this request to the SCP composer to compose CQ (Command queue) buffer
> > > via SCP_ISP_FRAME IPI command.
> > > - CQ buffer is frame registers set. If ISP registers should be updated
> > > per frame, these registers are configured in the CQ buffer, such as
> > > frame sequence number, DMA addresses and tuning ISP registers.
> > > - One frame request will be composed into one CQ buffer.Once CQ buffer
> > > is composed done and kernel driver will receive ISP_CMD_FRAME_ACK with
> > > its corresponding frame sequence number. Based on this, kernel driver
> > > knows which request is ready to be en-queued and save this with
> > > p1_dev->isp_ctx.composed_frame_id.
> >
> > Hmm, why do we need to save this in p1_dev->isp_ctx? Wouldn't we
> > already have a linked lists of requests that are composed and ready to
> > be enqueued? Also, the request itself would contain its frame ID
> > inside the driver request struct, right?
> >
>
> Below is current implementation for frame request en-queued.
> Before en-queued into HW by CQ, the request should be composed by SCP
> composer.
>
> a. mtk_cam_dev_req_try_queue()
> - Insert the request into p1_dev->running_job_list
> b. mtk_isp_req_enqueue()
> - Assign new next frame ID to this request.
> - Sending to SCP by workqueue
> - This request is ready to compose
> c. isp_tx_frame_worker()
> - Send request to SCP with sync. mode. by SCP_IPI_ISP_FRAME command
> - SCP composer will compose the buffer CQ for this request frame based
> on struct mtk_p1_frame_param which includes frame ID.
> - If scp_ipi_send() is returned, it means the request is composed done.
> Or
> d. isp_composer_handler()
> - If we received the ISP_CMD_FRAME_ACK for SCP_IPI_ISP_FRAME, we save
> the frame ID in p1_dev->composed_frame_seq_no which is sent in step C.
> - The request is composed done here.
> e. isp_irq_handle_sof()
> - In SOF timing, we will check there is any available composed CQ
> buffers by comparing composed & current de-queued frame ID.
>
> For p1_dev->running_job_list, we can't guarantee the requests are
> composed until the end of step c. For step e, we need to know how many
> available composed requests are ready to en-queued.
>
> Do you suggest we add another new link-list to save these requests in
> step c or we could update p1_dev->composed_frame_seq_no in step c and
> remove the implementation in step d[1]?

Okay, thanks to your explanation above I think I understood how the
hardware flow behaves and so I think we can indeed keep the
composed_frame_seq counter. Thanks!

>
> [1]. isp_composer_handler() is mandatory callback function for SCP
> sending API with sync mode design.
>
> static void isp_composer_handler(void *data, unsigned int len, void
> *priv)
> {
>         struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
>         struct mtk_isp_scp_p1_cmd *ipi_msg;
>
>         ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
>
>         if (ipi_msg->cmd_id != ISP_CMD_ACK)
>                 return;
>
>         if (ipi_msg->ack_info.cmd_id == ISP_CMD_FRAME_ACK) {
>                 atomic_set(&p1_dev->composed_frame_seq_no,
>                            ipi_msg->ack_info.frame_seq_no);
>                 dev_dbg(p1_dev->dev, "ack frame_num:%d\n",
>                         p1_dev->composed_frame_seq_no);
>         }
> }
>
> > > - The maximum number of CQ buffers in SCP is 3.
> > >
> > > 2. Buffer en-queue flow
> > > - In order to configure correct CQ buffer setting before next SQF event,
> > > it is depended on by MTK ISP P1 HW CQ mechanism.
> > > - The basic concept of CQ mechanism is loaded ISP CQ buffer settings
> > > when HW_PASS1_DON_ST is received which means DMA output is done.
> > > - Btw, the pre-condition of this, need to tell ISP HW which CQ buffer
> > > address is used. Otherwise, it will loaded one dummy CQ buffer to
> > > bypass.
> > > - So we will check available CQ buffers by comparing composed frame
> > > sequence number & dequeued frame sequence from ISP HW in SOF event.
> > > - If there are available CQ buffers, update the CQ base address to the
> > > next CQ buffer address based on current de-enqueue frame sequence
> > > number. So MTK ISP P1 HW will load this CQ buffer into HW when
> > > HW_PASS1_DON_ST is triggered which is before the next SOF.
> > > - So in next SOF event, ISP HW starts to output DMA buffers with this
> > > request until request is done.
> > > - But, for the first request, it is loaded into HW manually when
> > > streaming is on for better performance.
> > >
> > > 3. Buffer de-queue flow
> > > - We will use frame sequence number to decide which request is ready to
> > > de-queue.
> > > - We will save some important register setting from ISP HW when SOF is
> > > received. This is because the ISP HW starts to output the data with the
> > > corresponding settings, especially frame sequence number setting.
> >
> > Could you explain a bit more about these important register settings?
> > When does the hardware update the values in the register to new ones?
> > At SOF?
> >
>
> Sorry about my words.
> In the current implementation, we just save frame ID.
>

Ah, okay, makes sense. No worries. :)

>
> > > - When receiving SW_PASS1_DON_ST IRQ event, it means the DMA output is
> > > done. So we could call isp_deque_request_frame with frame sequence
> > > number to de-queue frame to VB2
> >
> > What's the difference between HW_PASS1_DON_ST and SW_PASS1_DON_ST?
> >
>
> This is explained above.
>
> > > - For AAO/AFO buffers, it has similar design concept. Sometimes, if only
> > > AAO/AFO non-request buffers are en-queued without request buffers at the
> > > same time, there will be no SW P1 done event for AAO/AFO DMA done.
> > > Needs to depend on other IRQ events, such as AAO/AFO_DONE_EVENT.
> >
> > Do we have a case like this? Wouldn't we normally always want to
> > bundle AAO/AFO buffers with frame buffers?
> >
>
> For upstream driver, we will remove non-request design.
>

I think we also talked about a thing related to this in the thread for
another patch from this series. Basically on Chrome OS we want to use
the upstream driver, so corresponding userspace changes might be
needed as well.

> > > - Due to CQ buffer number limitation, if we receive SW_PASS1_DONT_ST,
> > > we may try to send another request to SCP for composing.
> >
> > Okay, so basically in SW_PASS1_DONT_ST the CQ completed reading the CQ
> > buffers, right?
> >
>
> We expected the the life cycle of CQ buffer is same as frame request.
> So SW_PASS1_DON_ST is good timing to re-queue the next request to
> compose.
> For the CQ operations, we will explain later.
>
> > >
> > > Hopefully, my explanation is helpful for better understanding our
> > > implementation. If you still have any questions, please let me know.
> > >
> >
> > Yes, it's more clear now, thanks. Still some more comments above, though.
> >
> > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +   } else {
> > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > +           }
> > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > +   }
> > > >
> > > > The if and else blocks do almost the same things just in different order. Is
> > > > it really expected?
> > > >
> > >
> > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > any de-queued frame and update the next frame setting later.
> > > Normally, this is a corner case or system performance issue.
> >
> > So it sounds like HW_PASS1_DON_ST means that all data from current
> > frame has been written, right? If I understand your explanation above
> > correctly, that would mean following handling of each interrupt:
> >
> > HW_PASS1_DON_ST:
> >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > is this handled? does the CQ hardware automatically receive this event
> > from the ISP hadware?
> >  - return VB2 buffers,
> >  - complete requests.
> >
> > SOF_INT_ST:
> >  - send VSYNC event to userspace,
> >  - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> >  - reclaim CQ buffer and enqueue next frame to composing if available
> >
>
> Sorry for our implementation of HW_PASS1_DON_ST.
> It is confusing.
> Below is the revised version based on your conclusion.
> So in our new implemmenation, we just handle SOF_INT_ST &
> SW_PASS1_DON_ST events. We just add one warning message for
> HW_PASS1_DON_ST
>
> HW_PASS1_DON_ST:
> - CQ executes with next CQ buffer to prepare for next frame.
>
> SOF_INT_ST:
> - send VSYNC event to userspace,
> - program next CQ buffer to CQ,
>
> SW_PASS1_DON_ST:
> - reclaim CQ buffer and enqueue next frame to composing if available
> - return VB2 buffers,
> - complete requests.
>
> For CQ HW operations, it is listed below:
>
> a. The CQ buffer has two kinds of information
>  - Which ISP registers needs to be updated.
>  - Where the corresponding ISP register data to be read.
> b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> event periodically.
>  - Normally, if the ISP HW receives the completed frame and it will
> trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> -  So the CQ buffer loading is performed by ISP HW automatically.
> c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> to decide which CQ buffer is loaded.
>    - So we configure the next CQ base address in SOF.
> d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> and update the ISP register values into HW.
>    - SCP composer will compose one dummy CQ buffer and assign it to
> REG_CQ_THR0_BASEADDR of each CQ buffer.
>    - Dummy CQ buffer has no updated ISP registers comparing with other
> CQ buffers.
>    - With this design, if there is no updated new CQ buffer by driver
> which may be caused no en-queue frames from user space. The CQ HW will
> load dummy CQ buffer and do nothing.

Does the set of registers programmed by CQ include destination buffer
addresses to? If yes, we would end up overwriting previous frames if
no new buffers are provided.

> f. The CQ buffer loading is guaranteed by HW to finish before the next
> SOF.
>

Okay, thanks a lot for the explanation. This is much more clear now.

[snip]
> > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > >
> > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > or is about to become active. System PM ops would be called at system power
> > > > state change and the hardware might be both idle or active. Please also see
> > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > >
> > >
> > > Here is the new implementation. It should be clear to show the
> > > difference between system and runtime PM ops.
> > >
> > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > >                                 pm_runtime_force_resume)
> > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > NULL)
> > > };
> >
> > That's still not correct. In runtime suspend/resume ops we already are
> > not streaming anymore, because we call pm_runtime_get/put_*() when
> > starting and stopping streaming. In system suspend/resume ops we might
> > be streaming and that's when we need to stop the hardware and wait for
> > it to finish. Please implement these ops separately.
> >
> > Best regards,
> > Tomasz
>
>
> Ok, got your point.
> Below is the new implementation for your review.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Check ISP is streaming or not */
>         if (!p1_dev->cam_dev.streaming)
>                 goto done;

We would normally check here for pm_runtime_suspended(). Although they
both should be equivalent. Still, there is no need to call
pm_runtime_force_suspend() if the latter is true, so we could just
return 0 instantly.

>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "Cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
> done:
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;
>
>         return 0;
> }
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         /* Force ISP HW to resume if needed */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;

We should do this conditionally based on what pm_runtime_suspended()
returns. If it's non-zero then we can just return 0 instantly.

>
>         if (!p1_dev->cam_dev.streaming)
>                 return 0;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "Cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

Sorry, I guess I wasn't clear in my reply. It's not possible to get
this callback called if the device is already runtime suspended.

>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;

In this case the above call would always return non-zero, so the
behavior wouldn't be very good.

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-08-06  9:47               ` Tomasz Figa
  (?)
@ 2019-08-07  2:11                 ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07  2:11 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list

Hi, Tomasz:

On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > .Hi Jungo,
> > >
> > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> [snip]

I just keep some questions to be clarified.
[snip]

> > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +   } else {
> > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +           }
> > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > +   }
> > > > >
> > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > it really expected?
> > > > >
> > > >
> > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > any de-queued frame and update the next frame setting later.
> > > > Normally, this is a corner case or system performance issue.
> > >
> > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > frame has been written, right? If I understand your explanation above
> > > correctly, that would mean following handling of each interrupt:
> > >
> > > HW_PASS1_DON_ST:
> > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > is this handled? does the CQ hardware automatically receive this event
> > > from the ISP hadware?
> > >  - return VB2 buffers,
> > >  - complete requests.
> > >
> > > SOF_INT_ST:
> > >  - send VSYNC event to userspace,
> > >  - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > >
> >
> > Sorry for our implementation of HW_PASS1_DON_ST.
> > It is confusing.
> > Below is the revised version based on your conclusion.
> > So in our new implemmenation, we just handle SOF_INT_ST &
> > SW_PASS1_DON_ST events. We just add one warning message for
> > HW_PASS1_DON_ST
> >
> > HW_PASS1_DON_ST:
> > - CQ executes with next CQ buffer to prepare for next frame.
> >
> > SOF_INT_ST:
> > - send VSYNC event to userspace,
> > - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> > - reclaim CQ buffer and enqueue next frame to composing if available
> > - return VB2 buffers,
> > - complete requests.
> >
> > For CQ HW operations, it is listed below:
> >
> > a. The CQ buffer has two kinds of information
> >  - Which ISP registers needs to be updated.
> >  - Where the corresponding ISP register data to be read.
> > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > event periodically.
> >  - Normally, if the ISP HW receives the completed frame and it will
> > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > -  So the CQ buffer loading is performed by ISP HW automatically.
> > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > to decide which CQ buffer is loaded.
> >    - So we configure the next CQ base address in SOF.
> > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > and update the ISP register values into HW.
> >    - SCP composer will compose one dummy CQ buffer and assign it to
> > REG_CQ_THR0_BASEADDR of each CQ buffer.
> >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > CQ buffers.
> >    - With this design, if there is no updated new CQ buffer by driver
> > which may be caused no en-queue frames from user space. The CQ HW will
> > load dummy CQ buffer and do nothing.
> 
> Does the set of registers programmed by CQ include destination buffer
> addresses to? If yes, we would end up overwriting previous frames if
> no new buffers are provided.
> 

Yes, the buffer addresses are changed per frame request. We need to
compose CQ to include these DMA destination addresses. For your concern,
we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
increased due to no buffer for each DMA, the ISP HW doesn't output the
data to the corresponding DMA address.

Below is the simple descriptor of CQ buffer.
a. ISP registers in tuning buffer, including 3A registers.
b. All capture buffers informations.
   - DMA buffer destination address
   - FBC counter
c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
d. frame sequence number register

> > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > SOF.
> >
> 
> Okay, thanks a lot for the explanation. This is much more clear now.
> 
> [snip]
> > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > >
> > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > or is about to become active. System PM ops would be called at system power
> > > > > state change and the hardware might be both idle or active. Please also see
> > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > >
> > > >
> > > > Here is the new implementation. It should be clear to show the
> > > > difference between system and runtime PM ops.
> > > >
> > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > >                                 pm_runtime_force_resume)
> > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > NULL)
> > > > };
> > >
> > > That's still not correct. In runtime suspend/resume ops we already are
> > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > starting and stopping streaming. In system suspend/resume ops we might
> > > be streaming and that's when we need to stop the hardware and wait for
> > > it to finish. Please implement these ops separately.
> > >
> > > Best regards,
> > > Tomasz
> >
> >
> > Ok, got your point.
> > Below is the new implementation for your review.
> >
> > static int mtk_isp_pm_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Check ISP is streaming or not */
> >         if (!p1_dev->cam_dev.streaming)
> >                 goto done;
> 
> We would normally check here for pm_runtime_suspended(). Although they
> both should be equivalent. Still, there is no need to call
> pm_runtime_force_suspend() if the latter is true, so we could just
> return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}
[snip]

> > static int mtk_isp_pm_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Force ISP HW to resume if needed */
> >         ret = pm_runtime_force_resume(dev);
> >         if (ret)
> >                 return ret;
> 
> We should do this conditionally based on what pm_runtime_suspended()
> returns. If it's non-zero then we can just return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Force ISP HW to resume */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	/* Enable CMOS */
	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> Sorry, I guess I wasn't clear in my reply. It's not possible to get
> this callback called if the device is already runtime suspended.
> 

Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
it into mtk_isp_pm_* functions. If I still don't get your point, could
you kindly provide one sample driver for reference? Based on current
implementation, it is similar to below drivers.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113


static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> In this case the above call would always return non-zero, so the
> behavior wouldn't be very good.
> 

Same as above.

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "%s:enable clock\n", __func__);
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "failed to enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

Thanks for your further comments on these issues.

Best regards,

Jugno

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-07  2:11                 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07  2:11 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > .Hi Jungo,
> > >
> > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> [snip]

I just keep some questions to be clarified.
[snip]

> > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +   } else {
> > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +           }
> > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > +   }
> > > > >
> > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > it really expected?
> > > > >
> > > >
> > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > any de-queued frame and update the next frame setting later.
> > > > Normally, this is a corner case or system performance issue.
> > >
> > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > frame has been written, right? If I understand your explanation above
> > > correctly, that would mean following handling of each interrupt:
> > >
> > > HW_PASS1_DON_ST:
> > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > is this handled? does the CQ hardware automatically receive this event
> > > from the ISP hadware?
> > >  - return VB2 buffers,
> > >  - complete requests.
> > >
> > > SOF_INT_ST:
> > >  - send VSYNC event to userspace,
> > >  - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > >
> >
> > Sorry for our implementation of HW_PASS1_DON_ST.
> > It is confusing.
> > Below is the revised version based on your conclusion.
> > So in our new implemmenation, we just handle SOF_INT_ST &
> > SW_PASS1_DON_ST events. We just add one warning message for
> > HW_PASS1_DON_ST
> >
> > HW_PASS1_DON_ST:
> > - CQ executes with next CQ buffer to prepare for next frame.
> >
> > SOF_INT_ST:
> > - send VSYNC event to userspace,
> > - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> > - reclaim CQ buffer and enqueue next frame to composing if available
> > - return VB2 buffers,
> > - complete requests.
> >
> > For CQ HW operations, it is listed below:
> >
> > a. The CQ buffer has two kinds of information
> >  - Which ISP registers needs to be updated.
> >  - Where the corresponding ISP register data to be read.
> > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > event periodically.
> >  - Normally, if the ISP HW receives the completed frame and it will
> > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > -  So the CQ buffer loading is performed by ISP HW automatically.
> > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > to decide which CQ buffer is loaded.
> >    - So we configure the next CQ base address in SOF.
> > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > and update the ISP register values into HW.
> >    - SCP composer will compose one dummy CQ buffer and assign it to
> > REG_CQ_THR0_BASEADDR of each CQ buffer.
> >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > CQ buffers.
> >    - With this design, if there is no updated new CQ buffer by driver
> > which may be caused no en-queue frames from user space. The CQ HW will
> > load dummy CQ buffer and do nothing.
> 
> Does the set of registers programmed by CQ include destination buffer
> addresses to? If yes, we would end up overwriting previous frames if
> no new buffers are provided.
> 

Yes, the buffer addresses are changed per frame request. We need to
compose CQ to include these DMA destination addresses. For your concern,
we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
increased due to no buffer for each DMA, the ISP HW doesn't output the
data to the corresponding DMA address.

Below is the simple descriptor of CQ buffer.
a. ISP registers in tuning buffer, including 3A registers.
b. All capture buffers informations.
   - DMA buffer destination address
   - FBC counter
c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
d. frame sequence number register

> > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > SOF.
> >
> 
> Okay, thanks a lot for the explanation. This is much more clear now.
> 
> [snip]
> > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > >
> > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > or is about to become active. System PM ops would be called at system power
> > > > > state change and the hardware might be both idle or active. Please also see
> > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > >
> > > >
> > > > Here is the new implementation. It should be clear to show the
> > > > difference between system and runtime PM ops.
> > > >
> > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > >                                 pm_runtime_force_resume)
> > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > NULL)
> > > > };
> > >
> > > That's still not correct. In runtime suspend/resume ops we already are
> > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > starting and stopping streaming. In system suspend/resume ops we might
> > > be streaming and that's when we need to stop the hardware and wait for
> > > it to finish. Please implement these ops separately.
> > >
> > > Best regards,
> > > Tomasz
> >
> >
> > Ok, got your point.
> > Below is the new implementation for your review.
> >
> > static int mtk_isp_pm_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Check ISP is streaming or not */
> >         if (!p1_dev->cam_dev.streaming)
> >                 goto done;
> 
> We would normally check here for pm_runtime_suspended(). Although they
> both should be equivalent. Still, there is no need to call
> pm_runtime_force_suspend() if the latter is true, so we could just
> return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}
[snip]

> > static int mtk_isp_pm_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Force ISP HW to resume if needed */
> >         ret = pm_runtime_force_resume(dev);
> >         if (ret)
> >                 return ret;
> 
> We should do this conditionally based on what pm_runtime_suspended()
> returns. If it's non-zero then we can just return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Force ISP HW to resume */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	/* Enable CMOS */
	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> Sorry, I guess I wasn't clear in my reply. It's not possible to get
> this callback called if the device is already runtime suspended.
> 

Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
it into mtk_isp_pm_* functions. If I still don't get your point, could
you kindly provide one sample driver for reference? Based on current
implementation, it is similar to below drivers.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113


static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> In this case the above call would always return non-zero, so the
> behavior wouldn't be very good.
> 

Same as above.

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "%s:enable clock\n", __func__);
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "failed to enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

Thanks for your further comments on these issues.

Best regards,

Jugno

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-07  2:11                 ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07  2:11 UTC (permalink / raw)
  To: Tomasz Figa
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

Hi, Tomasz:

On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> Hi Jungo,
> 
> On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> >
> > Hi, Tomasz:
> >
> > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > .Hi Jungo,
> > >
> > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > >
> > > > Hi, Tomasz:
> > > >
> > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > Hi Jungo,
> > > > >
> > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> [snip]

I just keep some questions to be clarified.
[snip]

> > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +   } else {
> > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > +           }
> > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > +   }
> > > > >
> > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > it really expected?
> > > > >
> > > >
> > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > any de-queued frame and update the next frame setting later.
> > > > Normally, this is a corner case or system performance issue.
> > >
> > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > frame has been written, right? If I understand your explanation above
> > > correctly, that would mean following handling of each interrupt:
> > >
> > > HW_PASS1_DON_ST:
> > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > is this handled? does the CQ hardware automatically receive this event
> > > from the ISP hadware?
> > >  - return VB2 buffers,
> > >  - complete requests.
> > >
> > > SOF_INT_ST:
> > >  - send VSYNC event to userspace,
> > >  - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > >
> >
> > Sorry for our implementation of HW_PASS1_DON_ST.
> > It is confusing.
> > Below is the revised version based on your conclusion.
> > So in our new implemmenation, we just handle SOF_INT_ST &
> > SW_PASS1_DON_ST events. We just add one warning message for
> > HW_PASS1_DON_ST
> >
> > HW_PASS1_DON_ST:
> > - CQ executes with next CQ buffer to prepare for next frame.
> >
> > SOF_INT_ST:
> > - send VSYNC event to userspace,
> > - program next CQ buffer to CQ,
> >
> > SW_PASS1_DON_ST:
> > - reclaim CQ buffer and enqueue next frame to composing if available
> > - return VB2 buffers,
> > - complete requests.
> >
> > For CQ HW operations, it is listed below:
> >
> > a. The CQ buffer has two kinds of information
> >  - Which ISP registers needs to be updated.
> >  - Where the corresponding ISP register data to be read.
> > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > event periodically.
> >  - Normally, if the ISP HW receives the completed frame and it will
> > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > -  So the CQ buffer loading is performed by ISP HW automatically.
> > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > to decide which CQ buffer is loaded.
> >    - So we configure the next CQ base address in SOF.
> > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > and update the ISP register values into HW.
> >    - SCP composer will compose one dummy CQ buffer and assign it to
> > REG_CQ_THR0_BASEADDR of each CQ buffer.
> >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > CQ buffers.
> >    - With this design, if there is no updated new CQ buffer by driver
> > which may be caused no en-queue frames from user space. The CQ HW will
> > load dummy CQ buffer and do nothing.
> 
> Does the set of registers programmed by CQ include destination buffer
> addresses to? If yes, we would end up overwriting previous frames if
> no new buffers are provided.
> 

Yes, the buffer addresses are changed per frame request. We need to
compose CQ to include these DMA destination addresses. For your concern,
we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
increased due to no buffer for each DMA, the ISP HW doesn't output the
data to the corresponding DMA address.

Below is the simple descriptor of CQ buffer.
a. ISP registers in tuning buffer, including 3A registers.
b. All capture buffers informations.
   - DMA buffer destination address
   - FBC counter
c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
d. frame sequence number register

> > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > SOF.
> >
> 
> Okay, thanks a lot for the explanation. This is much more clear now.
> 
> [snip]
> > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > >
> > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > or is about to become active. System PM ops would be called at system power
> > > > > state change and the hardware might be both idle or active. Please also see
> > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > >
> > > >
> > > > Here is the new implementation. It should be clear to show the
> > > > difference between system and runtime PM ops.
> > > >
> > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > >                                 pm_runtime_force_resume)
> > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > NULL)
> > > > };
> > >
> > > That's still not correct. In runtime suspend/resume ops we already are
> > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > starting and stopping streaming. In system suspend/resume ops we might
> > > be streaming and that's when we need to stop the hardware and wait for
> > > it to finish. Please implement these ops separately.
> > >
> > > Best regards,
> > > Tomasz
> >
> >
> > Ok, got your point.
> > Below is the new implementation for your review.
> >
> > static int mtk_isp_pm_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Check ISP is streaming or not */
> >         if (!p1_dev->cam_dev.streaming)
> >                 goto done;
> 
> We would normally check here for pm_runtime_suspended(). Although they
> both should be equivalent. Still, there is no need to call
> pm_runtime_force_suspend() if the latter is true, so we could just
> return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Disable ISP's view finder and wait for TG idle */
	dev_dbg(dev, "cam suspend, disable VF\n");
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
					(val & TG_CS_MASK) == TG_IDLE_ST,
					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
	if (ret)
		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

	/* Disable CMOS */
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);

	/* Force ISP HW to idle */
	ret = pm_runtime_force_suspend(dev);
	if (ret)
		return ret;

	return 0;
}
[snip]

> > static int mtk_isp_pm_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         u32 val;
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         /* Force ISP HW to resume if needed */
> >         ret = pm_runtime_force_resume(dev);
> >         if (ret)
> >                 return ret;
> 
> We should do this conditionally based on what pm_runtime_suspended()
> returns. If it's non-zero then we can just return 0 instantly.
> 

Ok, here is the fixed version.

static int mtk_isp_pm_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	u32 val;
	int ret;

	dev_dbg(dev, "- %s\n", __func__);

	if (pm_runtime_suspended(dev))
		return 0;

	/* Force ISP HW to resume */
	ret = pm_runtime_force_resume(dev);
	if (ret)
		return ret;

	/* Enable CMOS */
	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);

	/* Enable VF */
	val = readl(p1_dev->regs + REG_TG_VF_CON);
	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_suspend(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> Sorry, I guess I wasn't clear in my reply. It's not possible to get
> this callback called if the device is already runtime suspended.
> 

Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
it into mtk_isp_pm_* functions. If I still don't get your point, could
you kindly provide one sample driver for reference? Based on current
implementation, it is similar to below drivers.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113


static int mtk_isp_runtime_suspend(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);

	dev_dbg(dev, "%s:disable clock\n", __func__);
	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);

	return 0;
}

[snip]

> > static int mtk_isp_runtime_resume(struct device *dev)
> > {
> >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >         int ret;
> >
> >         dev_dbg(dev, "- %s\n", __func__);
> >
> >         if (pm_runtime_suspended(dev))
> >                 return 0;
> 
> In this case the above call would always return non-zero, so the
> behavior wouldn't be very good.
> 

Same as above.

static int mtk_isp_runtime_resume(struct device *dev)
{
	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
	int ret;

	dev_dbg(dev, "%s:enable clock\n", __func__);
	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
	if (ret) {
		dev_err(dev, "failed to enable clock:%d\n", ret);
		return ret;
	}

	return 0;
}

Thanks for your further comments on these issues.

Best regards,

Jugno

> Best regards,
> Tomasz
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, v4, 0/4] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-08-07 12:47   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:47 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.59

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
  test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (4):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   31 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   39 +
 24 files changed, 4166 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,v4,0/4] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-08-07 12:47   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:47 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.59

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
  test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (4):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   31 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   39 +
 24 files changed, 4166 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h



^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, v4, 0/4] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-08-07 12:47   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:47 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.59

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.59
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
  test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.59
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.59
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (4):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   31 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   39 +
 24 files changed, 4166 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-08-07 12:47   ` [RFC,v4,0/4] " Jungo Lin
  (?)
@ 2019-08-07 12:48     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..fa2713acceca
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..fa2713acceca
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..fa2713acceca
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,2/4] dts: arm64: mt8183: Add ISP Pass 1 nodes
  2019-08-07 12:47   ` [RFC,v4,0/4] " Jungo Lin
  (?)
@ 2019-08-07 12:48     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 31 ++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 111866d36862..1692466c62fc 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -487,6 +487,37 @@
 			#clock-cells = <1>;
 		};
 
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
+
 		larb6: larb@1a001000 {
 			compatible = "mediatek,mt8183-smi-larb";
 			reg = <0 0x1a001000 0 0x1000>;
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,2/4] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 31 ++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 111866d36862..1692466c62fc 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -487,6 +487,37 @@
 			#clock-cells = <1>;
 		};
 
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
+
 		larb6: larb@1a001000 {
 			compatible = "mediatek,mt8183-smi-larb";
 			reg = <0 0x1a001000 0 0x1000>;
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,2/4] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 31 ++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 111866d36862..1692466c62fc 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -487,6 +487,37 @@
 			#clock-cells = <1>;
 		};
 
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp", "syscon";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
+
 		larb6: larb@1a001000 {
 			compatible = "mediatek,mt8183-smi-larb";
 			reg = <0 0x1a001000 0 0x1000>;
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v4, 3/4] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-08-07 12:47   ` [RFC,v4,0/4] " Jungo Lin
  (?)
@ 2019-08-07 12:48     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..8eabcd6a97bc 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,3/4] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..8eabcd6a97bc 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v4, 3/4] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..8eabcd6a97bc 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,4/4] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-08-07 12:47   ` [RFC,v4,0/4] " Jungo Lin
  (?)
@ 2019-08-07 12:48     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11076543/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 11 files changed, 3340 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..061e9ca6c456 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..8679e160ae7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..5625d694ea07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,622 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+	p1_dev->timestamp = ktime_get_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(p1_dev->dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(p1_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no,
+					      p1_dev->timestamp);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+					(val & TG_CS_MASK) == TG_IDLE_ST,
+					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+	if (ret)
+		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..52cd7e5f7e23
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u64 timestamp;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..229eb7425c0e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2066 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 u64 tstamp_soe, enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 tstamp_eof = ktime_get_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = tstamp_eof;
+		else
+			vb->timestamp = tstamp_soe;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no, u64 tstamp)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	cam_req->buf_count = vb2_request_buffer_cnt(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	vb2_request_queue(req);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+
+	if (!--req->buf_count)
+		mtk_cam_dev_req_try_queue(cam);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+	else
+		vq->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..e17e7f37afad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @buf_count: Buffer count in this media request.
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	unsigned int buf_count;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no, u64 tstamp);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,4/4] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11076543/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 11 files changed, 3340 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..061e9ca6c456 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..8679e160ae7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..5625d694ea07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,622 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+	p1_dev->timestamp = ktime_get_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(p1_dev->dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(p1_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no,
+					      p1_dev->timestamp);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+					(val & TG_CS_MASK) == TG_IDLE_ST,
+					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+	if (ret)
+		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..52cd7e5f7e23
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u64 timestamp;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..229eb7425c0e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2066 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 u64 tstamp_soe, enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 tstamp_eof = ktime_get_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = tstamp_eof;
+		else
+			vb->timestamp = tstamp_soe;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no, u64 tstamp)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	cam_req->buf_count = vb2_request_buffer_cnt(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	vb2_request_queue(req);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+
+	if (!--req->buf_count)
+		mtk_cam_dev_req_try_queue(cam);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+	else
+		vq->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..e17e7f37afad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @buf_count: Buffer count in this media request.
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	unsigned int buf_count;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no, u64 tstamp);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v4,4/4] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2019-08-07 12:48     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-07 12:48 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11076543/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  622 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   65 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2066 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  242 ++
 11 files changed, 3340 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..061e9ca6c456 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..8679e160ae7e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..5625d694ea07
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,622 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/platform_data/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+	p1_dev->timestamp = ktime_get_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(p1_dev->dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(p1_dev->dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no,
+					      p1_dev->timestamp);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+					(val & TG_CS_MASK) == TG_IDLE_ST,
+					USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+	if (ret)
+		dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..52cd7e5f7e23
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u64 timestamp;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..229eb7425c0e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2066 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 u64 tstamp_soe, enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 tstamp_eof = ktime_get_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = tstamp_eof;
+		else
+			vb->timestamp = tstamp_soe;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no, u64 tstamp)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, tstamp,
+					     VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	cam_req->buf_count = vb2_request_buffer_cnt(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	vb2_request_queue(req);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+
+	if (!--req->buf_count)
+		mtk_cam_dev_req_try_queue(cam);
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+	else
+		vq->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.link_notify = v4l2_pipeline_link_notify,
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..e17e7f37afad
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @buf_count: Buffer count in this media request.
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	unsigned int buf_count;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no, u64 tstamp);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
  2019-08-07  2:11                 ` Jungo Lin
  (?)
@ 2019-08-07 13:25                   ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-07 13:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元)

On Wed, Aug 7, 2019 at 11:11 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > > .Hi Jungo,
> > > >
> > > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi, Tomasz:
> > > > >
> > > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > > Hi Jungo,
> > > > > >
> > > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > [snip]
>
> I just keep some questions to be clarified.
> [snip]
>
> > > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +   } else {
> > > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +           }
> > > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > > +   }
> > > > > >
> > > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > > it really expected?
> > > > > >
> > > > >
> > > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > > any de-queued frame and update the next frame setting later.
> > > > > Normally, this is a corner case or system performance issue.
> > > >
> > > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > > frame has been written, right? If I understand your explanation above
> > > > correctly, that would mean following handling of each interrupt:
> > > >
> > > > HW_PASS1_DON_ST:
> > > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > > is this handled? does the CQ hardware automatically receive this event
> > > > from the ISP hadware?
> > > >  - return VB2 buffers,
> > > >  - complete requests.
> > > >
> > > > SOF_INT_ST:
> > > >  - send VSYNC event to userspace,
> > > >  - program next CQ buffer to CQ,
> > > >
> > > > SW_PASS1_DON_ST:
> > > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > > >
> > >
> > > Sorry for our implementation of HW_PASS1_DON_ST.
> > > It is confusing.
> > > Below is the revised version based on your conclusion.
> > > So in our new implemmenation, we just handle SOF_INT_ST &
> > > SW_PASS1_DON_ST events. We just add one warning message for
> > > HW_PASS1_DON_ST
> > >
> > > HW_PASS1_DON_ST:
> > > - CQ executes with next CQ buffer to prepare for next frame.
> > >
> > > SOF_INT_ST:
> > > - send VSYNC event to userspace,
> > > - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > > - reclaim CQ buffer and enqueue next frame to composing if available
> > > - return VB2 buffers,
> > > - complete requests.
> > >
> > > For CQ HW operations, it is listed below:
> > >
> > > a. The CQ buffer has two kinds of information
> > >  - Which ISP registers needs to be updated.
> > >  - Where the corresponding ISP register data to be read.
> > > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > > event periodically.
> > >  - Normally, if the ISP HW receives the completed frame and it will
> > > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > > -  So the CQ buffer loading is performed by ISP HW automatically.
> > > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > > to decide which CQ buffer is loaded.
> > >    - So we configure the next CQ base address in SOF.
> > > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > > and update the ISP register values into HW.
> > >    - SCP composer will compose one dummy CQ buffer and assign it to
> > > REG_CQ_THR0_BASEADDR of each CQ buffer.
> > >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > > CQ buffers.
> > >    - With this design, if there is no updated new CQ buffer by driver
> > > which may be caused no en-queue frames from user space. The CQ HW will
> > > load dummy CQ buffer and do nothing.
> >
> > Does the set of registers programmed by CQ include destination buffer
> > addresses to? If yes, we would end up overwriting previous frames if
> > no new buffers are provided.
> >
>
> Yes, the buffer addresses are changed per frame request. We need to
> compose CQ to include these DMA destination addresses. For your concern,
> we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
> increased due to no buffer for each DMA, the ISP HW doesn't output the
> data to the corresponding DMA address.
>
> Below is the simple descriptor of CQ buffer.
> a. ISP registers in tuning buffer, including 3A registers.
> b. All capture buffers informations.
>    - DMA buffer destination address
>    - FBC counter
> c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
> d. frame sequence number register
>

Okay, with the FBC counter it sounds fine. Thanks for clarifying.

> > > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > > SOF.
> > >
> >
> > Okay, thanks a lot for the explanation. This is much more clear now.
> >
> > [snip]
> > > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > > >
> > > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > > or is about to become active. System PM ops would be called at system power
> > > > > > state change and the hardware might be both idle or active. Please also see
> > > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > > >
> > > > >
> > > > > Here is the new implementation. It should be clear to show the
> > > > > difference between system and runtime PM ops.
> > > > >
> > > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > > >                                 pm_runtime_force_resume)
> > > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > > NULL)
> > > > > };
> > > >
> > > > That's still not correct. In runtime suspend/resume ops we already are
> > > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > > starting and stopping streaming. In system suspend/resume ops we might
> > > > be streaming and that's when we need to stop the hardware and wait for
> > > > it to finish. Please implement these ops separately.
> > > >
> > > > Best regards,
> > > > Tomasz
> > >
> > >
> > > Ok, got your point.
> > > Below is the new implementation for your review.
> > >
> > > static int mtk_isp_pm_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Check ISP is streaming or not */
> > >         if (!p1_dev->cam_dev.streaming)
> > >                 goto done;
> >
> > We would normally check here for pm_runtime_suspended(). Although they
> > both should be equivalent. Still, there is no need to call
> > pm_runtime_force_suspend() if the latter is true, so we could just
> > return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

What happens in this case? Is it safe to continue?

>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;

We should probably reenable the hardware if the above failed, so that
we hopefully end up in the same state as before the suspend.

>
>         return 0;
> }
> [snip]
>
> > > static int mtk_isp_pm_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Force ISP HW to resume if needed */
> > >         ret = pm_runtime_force_resume(dev);
> > >         if (ret)
> > >                 return ret;
> >
> > We should do this conditionally based on what pm_runtime_suspended()
> > returns. If it's non-zero then we can just return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Force ISP HW to resume */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > Sorry, I guess I wasn't clear in my reply. It's not possible to get
> > this callback called if the device is already runtime suspended.
> >
>
> Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
> it into mtk_isp_pm_* functions. If I still don't get your point, could
> you kindly provide one sample driver for reference?

The above implementation is okay, thanks. :)

> Based on current
> implementation, it is similar to below drivers.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113
>

The first one is an m2m device so it has slightly different rules -
the runtime PM is allowed to suspend between frames if the idle time
is long enough. The second one is a dummy driver for some fake i2c
bus, so it doesn't really have any meaningful implementation.

I think you could take a look at
https://elixir.bootlin.com/linux/v5.3-rc3/source/drivers/media/platform/exynos4-is/fimc-lite.c#L1550
, which is an online capture device too.

>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > In this case the above call would always return non-zero, so the
> > behavior wouldn't be very good.
> >
>
> Same as above.
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "%s:enable clock\n", __func__);
>         ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "failed to enable clock:%d\n", ret);
>                 return ret;
>         }
>
>         return 0;
> }

Makes sense, thanks!

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-07 13:25                   ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-07 13:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Frederic Chen (陳俊元),
	Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, ddavenport, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List

On Wed, Aug 7, 2019 at 11:11 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > > .Hi Jungo,
> > > >
> > > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi, Tomasz:
> > > > >
> > > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > > Hi Jungo,
> > > > > >
> > > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > [snip]
>
> I just keep some questions to be clarified.
> [snip]
>
> > > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +   } else {
> > > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +           }
> > > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > > +   }
> > > > > >
> > > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > > it really expected?
> > > > > >
> > > > >
> > > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > > any de-queued frame and update the next frame setting later.
> > > > > Normally, this is a corner case or system performance issue.
> > > >
> > > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > > frame has been written, right? If I understand your explanation above
> > > > correctly, that would mean following handling of each interrupt:
> > > >
> > > > HW_PASS1_DON_ST:
> > > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > > is this handled? does the CQ hardware automatically receive this event
> > > > from the ISP hadware?
> > > >  - return VB2 buffers,
> > > >  - complete requests.
> > > >
> > > > SOF_INT_ST:
> > > >  - send VSYNC event to userspace,
> > > >  - program next CQ buffer to CQ,
> > > >
> > > > SW_PASS1_DON_ST:
> > > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > > >
> > >
> > > Sorry for our implementation of HW_PASS1_DON_ST.
> > > It is confusing.
> > > Below is the revised version based on your conclusion.
> > > So in our new implemmenation, we just handle SOF_INT_ST &
> > > SW_PASS1_DON_ST events. We just add one warning message for
> > > HW_PASS1_DON_ST
> > >
> > > HW_PASS1_DON_ST:
> > > - CQ executes with next CQ buffer to prepare for next frame.
> > >
> > > SOF_INT_ST:
> > > - send VSYNC event to userspace,
> > > - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > > - reclaim CQ buffer and enqueue next frame to composing if available
> > > - return VB2 buffers,
> > > - complete requests.
> > >
> > > For CQ HW operations, it is listed below:
> > >
> > > a. The CQ buffer has two kinds of information
> > >  - Which ISP registers needs to be updated.
> > >  - Where the corresponding ISP register data to be read.
> > > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > > event periodically.
> > >  - Normally, if the ISP HW receives the completed frame and it will
> > > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > > -  So the CQ buffer loading is performed by ISP HW automatically.
> > > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > > to decide which CQ buffer is loaded.
> > >    - So we configure the next CQ base address in SOF.
> > > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > > and update the ISP register values into HW.
> > >    - SCP composer will compose one dummy CQ buffer and assign it to
> > > REG_CQ_THR0_BASEADDR of each CQ buffer.
> > >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > > CQ buffers.
> > >    - With this design, if there is no updated new CQ buffer by driver
> > > which may be caused no en-queue frames from user space. The CQ HW will
> > > load dummy CQ buffer and do nothing.
> >
> > Does the set of registers programmed by CQ include destination buffer
> > addresses to? If yes, we would end up overwriting previous frames if
> > no new buffers are provided.
> >
>
> Yes, the buffer addresses are changed per frame request. We need to
> compose CQ to include these DMA destination addresses. For your concern,
> we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
> increased due to no buffer for each DMA, the ISP HW doesn't output the
> data to the corresponding DMA address.
>
> Below is the simple descriptor of CQ buffer.
> a. ISP registers in tuning buffer, including 3A registers.
> b. All capture buffers informations.
>    - DMA buffer destination address
>    - FBC counter
> c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
> d. frame sequence number register
>

Okay, with the FBC counter it sounds fine. Thanks for clarifying.

> > > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > > SOF.
> > >
> >
> > Okay, thanks a lot for the explanation. This is much more clear now.
> >
> > [snip]
> > > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > > >
> > > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > > or is about to become active. System PM ops would be called at system power
> > > > > > state change and the hardware might be both idle or active. Please also see
> > > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > > >
> > > > >
> > > > > Here is the new implementation. It should be clear to show the
> > > > > difference between system and runtime PM ops.
> > > > >
> > > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > > >                                 pm_runtime_force_resume)
> > > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > > NULL)
> > > > > };
> > > >
> > > > That's still not correct. In runtime suspend/resume ops we already are
> > > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > > starting and stopping streaming. In system suspend/resume ops we might
> > > > be streaming and that's when we need to stop the hardware and wait for
> > > > it to finish. Please implement these ops separately.
> > > >
> > > > Best regards,
> > > > Tomasz
> > >
> > >
> > > Ok, got your point.
> > > Below is the new implementation for your review.
> > >
> > > static int mtk_isp_pm_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Check ISP is streaming or not */
> > >         if (!p1_dev->cam_dev.streaming)
> > >                 goto done;
> >
> > We would normally check here for pm_runtime_suspended(). Although they
> > both should be equivalent. Still, there is no need to call
> > pm_runtime_force_suspend() if the latter is true, so we could just
> > return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

What happens in this case? Is it safe to continue?

>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;

We should probably reenable the hardware if the above failed, so that
we hopefully end up in the same state as before the suspend.

>
>         return 0;
> }
> [snip]
>
> > > static int mtk_isp_pm_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Force ISP HW to resume if needed */
> > >         ret = pm_runtime_force_resume(dev);
> > >         if (ret)
> > >                 return ret;
> >
> > We should do this conditionally based on what pm_runtime_suspended()
> > returns. If it's non-zero then we can just return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Force ISP HW to resume */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > Sorry, I guess I wasn't clear in my reply. It's not possible to get
> > this callback called if the device is already runtime suspended.
> >
>
> Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
> it into mtk_isp_pm_* functions. If I still don't get your point, could
> you kindly provide one sample driver for reference?

The above implementation is okay, thanks. :)

> Based on current
> implementation, it is similar to below drivers.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113
>

The first one is an m2m device so it has slightly different rules -
the runtime PM is allowed to suspend between frames if the idle time
is long enough. The second one is a dummy driver for some fake i2c
bus, so it doesn't really have any meaningful implementation.

I think you could take a look at
https://elixir.bootlin.com/linux/v5.3-rc3/source/drivers/media/platform/exynos4-is/fimc-lite.c#L1550
, which is an online capture device too.

>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > In this case the above call would always return non-zero, so the
> > behavior wouldn't be very good.
> >
>
> Same as above.
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "%s:enable clock\n", __func__);
>         ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "failed to enable clock:%d\n", ret);
>                 return ret;
>         }
>
>         return 0;
> }

Makes sense, thanks!

Best regards,
Tomasz

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver
@ 2019-08-07 13:25                   ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2019-08-07 13:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: devicetree, Sean Cheng (鄭昇弘),
	Mauro Carvalho Chehab, Rynn Wu (吳育恩),
	srv_heupstream, Rob Herring, Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Hans Verkuil, Matthias Brugger, Sj Huang,
	moderated list:ARM/Mediatek SoC support, Laurent Pinchart,
	ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Linux Media Mailing List

On Wed, Aug 7, 2019 at 11:11 AM Jungo Lin <jungo.lin@mediatek.com> wrote:
>
> Hi, Tomasz:
>
> On Tue, 2019-08-06 at 18:47 +0900, Tomasz Figa wrote:
> > Hi Jungo,
> >
> > On Fri, Jul 26, 2019 at 4:24 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > >
> > > Hi, Tomasz:
> > >
> > > On Thu, 2019-07-25 at 18:23 +0900, Tomasz Figa wrote:
> > > > .Hi Jungo,
> > > >
> > > > On Sat, Jul 20, 2019 at 6:58 PM Jungo Lin <jungo.lin@mediatek.com> wrote:
> > > > >
> > > > > Hi, Tomasz:
> > > > >
> > > > > On Wed, 2019-07-10 at 18:56 +0900, Tomasz Figa wrote:
> > > > > > Hi Jungo,
> > > > > >
> > > > > > On Tue, Jun 11, 2019 at 11:53:42AM +0800, Jungo Lin wrote:
> > [snip]
>
> I just keep some questions to be clarified.
> [snip]
>
> > > > > > > +           isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +           isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +   } else {
> > > > > > > +           if (irq_status & SOF_INT_ST) {
> > > > > > > +                   isp_dev->current_frame = hw_frame_num;
> > > > > > > +                   isp_dev->meta0_vb2_index = meta0_vb2_index;
> > > > > > > +                   isp_dev->meta1_vb2_index = meta1_vb2_index;
> > > > > > > +           }
> > > > > > > +           irq_handle_notify_event(isp_dev, irq_status, dma_status, 1);
> > > > > > > +   }
> > > > > >
> > > > > > The if and else blocks do almost the same things just in different order. Is
> > > > > > it really expected?
> > > > > >
> > > > >
> > > > > If we receive HW_PASS1_DON_ST & SOF_INT_ST IRQ events at the same time,
> > > > > the correct sequence should be handle HW_PASS1_DON_ST firstly to check
> > > > > any de-queued frame and update the next frame setting later.
> > > > > Normally, this is a corner case or system performance issue.
> > > >
> > > > So it sounds like HW_PASS1_DON_ST means that all data from current
> > > > frame has been written, right? If I understand your explanation above
> > > > correctly, that would mean following handling of each interrupt:
> > > >
> > > > HW_PASS1_DON_ST:
> > > >  - CQ executes with next CQ buffer to prepare for next frame. <- how
> > > > is this handled? does the CQ hardware automatically receive this event
> > > > from the ISP hadware?
> > > >  - return VB2 buffers,
> > > >  - complete requests.
> > > >
> > > > SOF_INT_ST:
> > > >  - send VSYNC event to userspace,
> > > >  - program next CQ buffer to CQ,
> > > >
> > > > SW_PASS1_DON_ST:
> > > >  - reclaim CQ buffer and enqueue next frame to composing if available
> > > >
> > >
> > > Sorry for our implementation of HW_PASS1_DON_ST.
> > > It is confusing.
> > > Below is the revised version based on your conclusion.
> > > So in our new implemmenation, we just handle SOF_INT_ST &
> > > SW_PASS1_DON_ST events. We just add one warning message for
> > > HW_PASS1_DON_ST
> > >
> > > HW_PASS1_DON_ST:
> > > - CQ executes with next CQ buffer to prepare for next frame.
> > >
> > > SOF_INT_ST:
> > > - send VSYNC event to userspace,
> > > - program next CQ buffer to CQ,
> > >
> > > SW_PASS1_DON_ST:
> > > - reclaim CQ buffer and enqueue next frame to composing if available
> > > - return VB2 buffers,
> > > - complete requests.
> > >
> > > For CQ HW operations, it is listed below:
> > >
> > > a. The CQ buffer has two kinds of information
> > >  - Which ISP registers needs to be updated.
> > >  - Where the corresponding ISP register data to be read.
> > > b. The CQ buffer loading procedure is triggered by HW_PASS1_DONT_ST IRQ
> > > event periodically.
> > >  - Normally, if the ISP HW receives the completed frame and it will
> > > trigger W_PASS1_DONT_ST IRQ and perform CQ buffer loading immediately.
> > > -  So the CQ buffer loading is performed by ISP HW automatically.
> > > c. The ISP HW will read CQ base address register(REG_CQ_THR0_BASEADDR)
> > > to decide which CQ buffer is loaded.
> > >    - So we configure the next CQ base address in SOF.
> > > d. For CQ buffer loading, CQ will read the ISP registers from CQ buffer
> > > and update the ISP register values into HW.
> > >    - SCP composer will compose one dummy CQ buffer and assign it to
> > > REG_CQ_THR0_BASEADDR of each CQ buffer.
> > >    - Dummy CQ buffer has no updated ISP registers comparing with other
> > > CQ buffers.
> > >    - With this design, if there is no updated new CQ buffer by driver
> > > which may be caused no en-queue frames from user space. The CQ HW will
> > > load dummy CQ buffer and do nothing.
> >
> > Does the set of registers programmed by CQ include destination buffer
> > addresses to? If yes, we would end up overwriting previous frames if
> > no new buffers are provided.
> >
>
> Yes, the buffer addresses are changed per frame request. We need to
> compose CQ to include these DMA destination addresses. For your concern,
> we have DMA flow buffer control (FBC) in HW. If there is no FBC counter
> increased due to no buffer for each DMA, the ISP HW doesn't output the
> data to the corresponding DMA address.
>
> Below is the simple descriptor of CQ buffer.
> a. ISP registers in tuning buffer, including 3A registers.
> b. All capture buffers informations.
>    - DMA buffer destination address
>    - FBC counter
> c. Some specif ISP registers for meta DMAs, such as LCE or LMVO.
> d. frame sequence number register
>

Okay, with the FBC counter it sounds fine. Thanks for clarifying.

> > > f. The CQ buffer loading is guaranteed by HW to finish before the next
> > > SOF.
> > >
> >
> > Okay, thanks a lot for the explanation. This is much more clear now.
> >
> > [snip]
> > > > > > > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > > > > +   SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_suspend, mtk_isp_resume)
> > > > > > > +   SET_RUNTIME_PM_OPS(mtk_isp_suspend, mtk_isp_resume, NULL)
> > > > > >
> > > > > > For V4L2 drivers system and runtime PM ops would normally be completely
> > > > > > different. Runtime PM ops would be called when the hardware is idle already
> > > > > > or is about to become active. System PM ops would be called at system power
> > > > > > state change and the hardware might be both idle or active. Please also see
> > > > > > my comments to mtk_isp_suspend() and mtk_isp_resume() above.
> > > > > >
> > > > >
> > > > > Here is the new implementation. It should be clear to show the
> > > > > difference between system and runtime PM ops.
> > > > >
> > > > > static const struct dev_pm_ops mtk_isp_pm_ops = {
> > > > >         SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
> > > > >                                 pm_runtime_force_resume)
> > > > >         SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > > > > NULL)
> > > > > };
> > > >
> > > > That's still not correct. In runtime suspend/resume ops we already are
> > > > not streaming anymore, because we call pm_runtime_get/put_*() when
> > > > starting and stopping streaming. In system suspend/resume ops we might
> > > > be streaming and that's when we need to stop the hardware and wait for
> > > > it to finish. Please implement these ops separately.
> > > >
> > > > Best regards,
> > > > Tomasz
> > >
> > >
> > > Ok, got your point.
> > > Below is the new implementation for your review.
> > >
> > > static int mtk_isp_pm_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Check ISP is streaming or not */
> > >         if (!p1_dev->cam_dev.streaming)
> > >                 goto done;
> >
> > We would normally check here for pm_runtime_suspended(). Although they
> > both should be equivalent. Still, there is no need to call
> > pm_runtime_force_suspend() if the latter is true, so we could just
> > return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Disable ISP's view finder and wait for TG idle */
>         dev_dbg(dev, "cam suspend, disable VF\n");
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>         ret = readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>                                         (val & TG_CS_MASK) == TG_IDLE_ST,
>                                         USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>         if (ret)
>                 dev_warn(dev, "can't stop HW:%d:0x%x\n", ret, val);

What happens in this case? Is it safe to continue?

>
>         /* Disable CMOS */
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Force ISP HW to idle */
>         ret = pm_runtime_force_suspend(dev);
>         if (ret)
>                 return ret;

We should probably reenable the hardware if the above failed, so that
we hopefully end up in the same state as before the suspend.

>
>         return 0;
> }
> [snip]
>
> > > static int mtk_isp_pm_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         u32 val;
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         /* Force ISP HW to resume if needed */
> > >         ret = pm_runtime_force_resume(dev);
> > >         if (ret)
> > >                 return ret;
> >
> > We should do this conditionally based on what pm_runtime_suspended()
> > returns. If it's non-zero then we can just return 0 instantly.
> >
>
> Ok, here is the fixed version.
>
> static int mtk_isp_pm_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         u32 val;
>         int ret;
>
>         dev_dbg(dev, "- %s\n", __func__);
>
>         if (pm_runtime_suspended(dev))
>                 return 0;
>
>         /* Force ISP HW to resume */
>         ret = pm_runtime_force_resume(dev);
>         if (ret)
>                 return ret;
>
>         /* Enable CMOS */
>         dev_dbg(dev, "cam resume, enable CMOS/VF\n");
>         val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>         writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>
>         /* Enable VF */
>         val = readl(p1_dev->regs + REG_TG_VF_CON);
>         writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_suspend(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > Sorry, I guess I wasn't clear in my reply. It's not possible to get
> > this callback called if the device is already runtime suspended.
> >
>
> Ok, got it. Need to remove pm_runtime_suspended(dev) checking and move
> it into mtk_isp_pm_* functions. If I still don't get your point, could
> you kindly provide one sample driver for reference?

The above implementation is okay, thanks. :)

> Based on current
> implementation, it is similar to below drivers.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/mtk-mdp/mtk_mdp_core.c#L255
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/exynos4-is/fimc-is-i2c.c#L113
>

The first one is an m2m device so it has slightly different rules -
the runtime PM is allowed to suspend between frames if the idle time
is long enough. The second one is a dummy driver for some fake i2c
bus, so it doesn't really have any meaningful implementation.

I think you could take a look at
https://elixir.bootlin.com/linux/v5.3-rc3/source/drivers/media/platform/exynos4-is/fimc-lite.c#L1550
, which is an online capture device too.

>
> static int mtk_isp_runtime_suspend(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>
>         dev_dbg(dev, "%s:disable clock\n", __func__);
>         clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>
>         return 0;
> }
>
> [snip]
>
> > > static int mtk_isp_runtime_resume(struct device *dev)
> > > {
> > >         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > >         int ret;
> > >
> > >         dev_dbg(dev, "- %s\n", __func__);
> > >
> > >         if (pm_runtime_suspended(dev))
> > >                 return 0;
> >
> > In this case the above call would always return non-zero, so the
> > behavior wouldn't be very good.
> >
>
> Same as above.
>
> static int mtk_isp_runtime_resume(struct device *dev)
> {
>         struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>         int ret;
>
>         dev_dbg(dev, "%s:enable clock\n", __func__);
>         ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
>         if (ret) {
>                 dev_err(dev, "failed to enable clock:%d\n", ret);
>                 return ret;
>         }
>
>         return 0;
> }

Makes sense, thanks!

Best regards,
Tomasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-08-07 12:48     ` Jungo Lin
  (?)
@ 2019-08-21 19:47       ` Rob Herring
  -1 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 19:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {
> +			compatible = "mediatek,mt8183-camisp", "syscon";

syscon doesn't seem appropriate nor is it documented.

> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-21 19:47       ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 19:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {
> +			compatible = "mediatek,mt8183-camisp", "syscon";

syscon doesn't seem appropriate nor is it documented.

> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-21 19:47       ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 19:47 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {
> +			compatible = "mediatek,mt8183-camisp", "syscon";

syscon doesn't seem appropriate nor is it documented.

> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-08-07 12:48     ` Jungo Lin
  (?)
@ 2019-08-21 20:17         ` Rob Herring
  -1 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 20:17 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu-NuS5LvNUpcJWk0Htik3J/w,
	frankie.chiu-NuS5LvNUpcJWk0Htik3J/w,
	laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw,
	Rynn.Wu-NuS5LvNUpcJWk0Htik3J/w, suleiman-F7+t8E8rja9g9hUCZPvPmw,
	Jerry-ch.Chen-NuS5LvNUpcJWk0Htik3J/w,
	frederic.chen-NuS5LvNUpcJWk0Htik3J/w,
	linux-media-u79uwXL29TY76Z2rM5mHXA,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	hverkuil-cisco-qWit8jRvyhVmR6Xm/wNWPw,
	shik-F7+t8E8rja9g9hUCZPvPmw, yuzhao-F7+t8E8rja9g9hUCZPvPmw,
	linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	matthias.bgg-Re5JQEeQqe8AvxtiuMwx3w,
	mchehab-DgEjT+Ai2ygdnm+yROfE0A,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	Sean.Cheng-NuS5LvNUpcJWk0Htik3J/w,
	srv_heupstream-NuS5LvNUpcJWk0Htik3J/w,
	sj.huang-NuS5LvNUpcJWk0Htik3J/w, tfiga-F7+t8E8rja9g9hUCZPvPmw,
	zwisler-F7+t8E8rja9g9hUCZPvPmw,
	ddavenport-F7+t8E8rja9g9hUCZPvPmw

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {

Also, you can remove 2 levels of indentation here.

> +			compatible = "mediatek,mt8183-camisp", "syscon";
> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-21 20:17         ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 20:17 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {

Also, you can remove 2 levels of indentation here.

> +			compatible = "mediatek,mt8183-camisp", "syscon";
> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-21 20:17         ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-08-21 20:17 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..fa2713acceca
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,73 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp : The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +
> +Example:
> +SoC specific DT entry:
> +
> +		camisp: camisp@1a000000 {

Also, you can remove 2 levels of indentation here.

> +			compatible = "mediatek,mt8183-camisp", "syscon";
> +			reg = <0 0x1a000000 0 0x1000>,
> +					<0 0x1a003000 0 0x1000>,
> +					<0 0x1a004000 0 0x2000>,
> +					<0 0x1a006000 0 0x2000>,
> +					<0 0x1a008000 0 0x2000>;
> +			reg-names = "cam_sys",
> +					"cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +			interrupt-names = "cam_uni",
> +					"cam_a",
> +					"cam_b",
> +					"cam_c";
> +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +			clocks = <&camsys CLK_CAM_CAM>,
> +					<&camsys CLK_CAM_CAMTG>;
> +			clock-names = "camsys_cam_cgpdn",
> +					"camsys_camtg_cgpdn";
> +			mediatek,larb = <&larb3>,
> +					<&larb6>;
> +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +			mediatek,scp = <&scp>;
> +		};
> -- 
> 2.18.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-08-21 19:47       ` Rob Herring
  (?)
@ 2019-08-22 12:47         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:47 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

Hi, Rob:

On Wed, 2019-08-21 at 14:47 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> 
> syscon doesn't seem appropriate nor is it documented.
> 

Thanks for your comment.
I will remove "syscon" in next patch.

Best regards,

Jungo

> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-22 12:47         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:47 UTC (permalink / raw)
  To: Rob Herring
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

Hi, Rob:

On Wed, 2019-08-21 at 14:47 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> 
> syscon doesn't seem appropriate nor is it documented.
> 

Thanks for your comment.
I will remove "syscon" in next patch.

Best regards,

Jungo

> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-22 12:47         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:47 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

Hi, Rob:

On Wed, 2019-08-21 at 14:47 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> 
> syscon doesn't seem appropriate nor is it documented.
> 

Thanks for your comment.
I will remove "syscon" in next patch.

Best regards,

Jungo

> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-08-21 20:17         ` Rob Herring
  (?)
@ 2019-08-22 12:48           ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:48 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

Hi, Rob:
On Wed, 2019-08-21 at 15:17 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> 
> Also, you can remove 2 levels of indentation here.
> 

Ok, got it.
We will change to use one level of indentation.

Thanks,

Jungo

> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-22 12:48           ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:48 UTC (permalink / raw)
  To: Rob Herring
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

Hi, Rob:
On Wed, 2019-08-21 at 15:17 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> 
> Also, you can remove 2 levels of indentation here.
> 

Ok, got it.
We will change to use one level of indentation.

Thanks,

Jungo

> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-08-22 12:48           ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-08-22 12:48 UTC (permalink / raw)
  To: Rob Herring
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

Hi, Rob:
On Wed, 2019-08-21 at 15:17 -0500, Rob Herring wrote:
> On Wed, Aug 07, 2019 at 08:48:00PM +0800, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
> >  1 file changed, 73 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..fa2713acceca
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,73 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp : The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +		camisp: camisp@1a000000 {
> 
> Also, you can remove 2 levels of indentation here.
> 

Ok, got it.
We will change to use one level of indentation.

Thanks,

Jungo

> > +			compatible = "mediatek,mt8183-camisp", "syscon";
> > +			reg = <0 0x1a000000 0 0x1000>,
> > +					<0 0x1a003000 0 0x1000>,
> > +					<0 0x1a004000 0 0x2000>,
> > +					<0 0x1a006000 0 0x2000>,
> > +					<0 0x1a008000 0 0x2000>;
> > +			reg-names = "cam_sys",
> > +					"cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +			interrupt-names = "cam_uni",
> > +					"cam_a",
> > +					"cam_b",
> > +					"cam_c";
> > +			iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +			clocks = <&camsys CLK_CAM_CAM>,
> > +					<&camsys CLK_CAM_CAMTG>;
> > +			clock-names = "camsys_cam_cgpdn",
> > +					"camsys_camtg_cgpdn";
> > +			mediatek,larb = <&larb3>,
> > +					<&larb6>;
> > +			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +			mediatek,scp = <&scp>;
> > +		};
> > -- 
> > 2.18.0
> > 



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, v5, 0/5] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
  2019-04-02 10:04   ` Jungo Lin
@ 2019-09-02  7:51   ` Jungo Lin
  2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
                     ` (14 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.67

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 5:
 - Fixed Rob's comment on dt-binding format
 - Fix Tomasz's comment in mtk_isp_pm_suspend function
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
   and new timestamp type in driver
 - Fix buffer en-queue timing issue in v4
 - Remove default link_notify callback function in mtk_cam_media_ops

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Align and pack IPI comamnd structures for EC ROM size shrink
 
version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (5):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: videodev2.h: Add new boottime timestamp type
  media: pixfmt: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 Documentation/media/uapi/v4l/buffer.rst       |   11 +-
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   30 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   41 +
 25 files changed, 4206 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,v5,0/5] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-09-02  7:51   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.67

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 5:
 - Fixed Rob's comment on dt-binding format
 - Fix Tomasz's comment in mtk_isp_pm_suspend function
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
   and new timestamp type in driver
 - Fix buffer en-queue timing issue in v4
 - Remove default link_notify callback function in mtk_cam_media_ops

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Align and pack IPI comamnd structures for EC ROM size shrink
 
version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (5):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: videodev2.h: Add new boottime timestamp type
  media: pixfmt: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 Documentation/media/uapi/v4l/buffer.rst       |   11 +-
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   30 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   41 +
 25 files changed, 4206 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 


^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC, v5, 0/5] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-09-02  7:51   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hello,

This RFC patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS. It's the first time Mediatek develops
ISP kernel drivers based on V4L2 and media controller framework.
I posted the main part of the ISP Pass 1 driver as RFC to discuss
first and would like some review comments on the overall architecture
of the driver.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 4 is the heart of ISP P1 driver. It handles the ISP
  HW configuration. Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP communication
  to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor & sen-inf sub-devices which are implemented
in below patch[1][2][3]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[6] & IOMMU[7] patchsets

/usr/bin/media-ctl -p -d /dev/media1

Media controller API version 4.19.59

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.67

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf.mipi-csi":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video25
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video26
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video27
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video28
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video29
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video30
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf.mipi-csi (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov5695 2-0036":0 []
	pad1: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov2685 4-003c":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov5695 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
	pad0: Source
		[fmt:SBGGR10_1X10/2592x1944 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":0 []

- entity 73: ov2685 4-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev8
	pad0: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "1a040000.seninf.mipi-csi":1 []

===========
= history =
===========

version 5:
 - Fixed Rob's comment on dt-binding format
 - Fix Tomasz's comment in mtk_isp_pm_suspend function
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
   and new timestamp type in driver
 - Fix buffer en-queue timing issue in v4
 - Remove default link_notify callback function in mtk_cam_media_ops

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Align and pack IPI comamnd structures for EC ROM size shrink
 
version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3 patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend MTK proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI comamnd structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. BACKPORT: FROMLIST: platform: mtk-isp: Add Mediatek sensor interface driver
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1482517

[2]. WIP: media: ov5695: support ov5695 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614887

[3]. WIP: media: ov2685: support ov2685 sensor in mt8183
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1614885

[4]. media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
https://patchwork.linuxtv.org/cover/56778/

[5]. [RFC,V2,4/6] platform: mtk-isp: Add Mediatek DIP driver
https://patchwork.linuxtv.org/patch/57472/

[6]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11076543/

[7]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/10984739/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=28be49b4e9d72c5866188cf5ba408541c665c921

Note 1.
This testing depends on the above seninf & sensors patches[1][2][3].

Note 2.
The current failure items are related to Mediatek seninf driver which
is under developing in other patchset.[1]

/usr/bin/v4l2-compliance -m /dev/media1

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video25: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video26:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video26 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video26: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video27:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video27 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video27: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video28:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video28 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video28: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video29:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video29 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video29: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video30:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video30 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video30: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video31:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video31 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK

Total for mtk-cam-p1 device /dev/video31: 45, Succeeded: 45, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev5:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x0300004f
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004d: from remote pad 0x100003d of entity '1a040000.seninf.mipi-csi': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev5 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev5: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev6:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000051
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf.mipi-csi
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov5695 2-0036': Data
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004b: from remote pad 0x100004a of entity 'ov2685 4-003c': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004d: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev6 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(381): s_fmt.format.code == ~0U
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev6: 125, Succeeded: 101, Failed: 24, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev7:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000053
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov5695 2-0036
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev7 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev7: 48, Succeeded: 48, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev8:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000055
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : ov2685 4-003c
	Function         : Camera Sensor
	Pad 0x0100004a   : 0: Source
	  Link 0x0200004b: to remote pad 0x100003a of entity '1a040000.seninf.mipi-csi': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev8 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev8: 48, Succeeded: 48, Failed: 0, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media1: 668, Succeeded: 644, Failed: 24, Warnings: 0

=========================================================================================

Jungo Lin (5):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: videodev2.h: Add new boottime timestamp type
  media: pixfmt: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   73 +
 Documentation/media/uapi/v4l/buffer.rst       |   11 +-
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |    8 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   30 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   41 +
 25 files changed, 4206 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [RFC,v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
  (?)
@ 2019-09-02  7:51     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..e156f01747d0
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+	};
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..e156f01747d0
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+	};
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..e156f01747d0
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,73 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp : The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+	};
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
  2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
  (?)
@ 2019-09-02  7:51     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 66aaa07f6cec..0d607342d4f1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -505,5 +505,35 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
 	};
 };
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 66aaa07f6cec..0d607342d4f1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -505,5 +505,35 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
 	};
 };
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 66aaa07f6cec..0d607342d4f1 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -505,5 +505,35 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+		};
 	};
 };
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 3/5] media: videodev2.h: Add new boottime timestamp type
  2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
  (?)
@ 2019-09-02  7:51     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 1cbd9cde57f3..9e636f4118f5 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -649,13 +649,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..a4fd271348e7 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1043,6 +1043,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 1cbd9cde57f3..9e636f4118f5 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -649,13 +649,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..a4fd271348e7 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1043,6 +1043,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 1cbd9cde57f3..9e636f4118f5 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -649,13 +649,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda76..a4fd271348e7 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1043,6 +1043,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v5, 4/5] media: pixfmt: Add Mediatek ISP P1 image & meta formats
  2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
  (?)
@ 2019-09-02  7:51     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a4fd271348e7..e515e681838c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 4/5] media: pixfmt: Add Mediatek ISP P1 image & meta formats
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a4fd271348e7..e515e681838c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v5, 4/5] media: pixfmt: Add Mediatek ISP P1 image & meta formats
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 Documentation/media/uapi/v4l/pixfmt-rgb.rst   |   8 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  37 ++++++
 include/uapi/linux/videodev2.h                |  39 +++++++
 11 files changed, 722 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-rgb.rst b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
index 48ab80024835..1ba260c84083 100644
--- a/Documentation/media/uapi/v4l/pixfmt-rgb.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-rgb.rst
@@ -28,3 +28,11 @@ RGB Formats
     pixfmt-srggb12p
     pixfmt-srggb14p
     pixfmt-srggb16
+    pixfmt-pixfmt-mtisp-srggb8
+    pixfmt-pixfmt-mtisp-srggb10
+    pixfmt-pixfmt-mtisp-srggb12
+    pixfmt-pixfmt-mtisp-srggb14
+    pixfmt-pixfmt-mtisp-srggb8f
+    pixfmt-pixfmt-mtisp-srggb10f
+    pixfmt-pixfmt-mtisp-srggb12f
+    pixfmt-pixfmt-mtisp-srggb14f
\ No newline at end of file
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba6..451dada2146d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1293,6 +1293,38 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_KONICA420:	descr = "GSPCA KONICA420"; break;
 	case V4L2_PIX_FMT_HSV24:	descr = "24-bit HSV 8-8-8"; break;
 	case V4L2_PIX_FMT_HSV32:	descr = "32-bit XHSV 8-8-8-8"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8: descr = "8-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8: descr = "8-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8: descr = "8-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8: descr = "8-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10: descr = "10-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10: descr = "10-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10: descr = "10-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10: descr = "10-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12: descr = "12-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12: descr = "12-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12: descr = "12-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12: descr = "12-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14: descr = "14-bit Bayer BGGR MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14: descr = "14-bit Bayer GBRG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14: descr = "14-bit Bayer GRBG MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14: descr = "14-bit Bayer RGGB MTISP Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F: descr = "8-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG8F: descr = "8-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG8F: descr = "8-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB8F: descr = "8-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F: descr = "10-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG10F: descr = "10-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG10F: descr = "10-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB10F: descr = "10-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F: descr = "12-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG12F: descr = "12-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG12F: descr = "12-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB12F: descr = "12-bit Full-G Bayer RGGB Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F: descr = "14-bit Full-G Bayer BGGR Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGBRG14F: descr = "14-bit Full-G Bayer GBRG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SGRBG14F: descr = "14-bit Full-G Bayer GRBG Packed"; break;
+	case V4L2_PIX_FMT_MTISP_SRGGB14F: descr = "14-bit Full-G Bayer RGGB Packed"; break;
 	case V4L2_SDR_FMT_CU8:		descr = "Complex U8"; break;
 	case V4L2_SDR_FMT_CU16LE:	descr = "Complex U16LE"; break;
 	case V4L2_SDR_FMT_CS8:		descr = "Complex S8"; break;
@@ -1308,6 +1340,11 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
 	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
+	case V4L2_META_FMT_MTISP_3A:	descr = "AE/AWB Histogram"; break;
+	case V4L2_META_FMT_MTISP_AF:	descr = "AF Histogram"; break;
+	case V4L2_META_FMT_MTISP_LCS:	descr = "Local Contrast Enhancement Stat"; break;
+	case V4L2_META_FMT_MTISP_LMV:	descr = "Local Motion Vector Histogram"; break;
+	case V4L2_META_FMT_MTISP_PARAMS: descr = "MTK ISP Tuning Metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a4fd271348e7..e515e681838c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -728,6 +728,40 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_IPU3_SGRBG10	v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */
 #define V4L2_PIX_FMT_IPU3_SRGGB10	v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */
 
+/* Vendor specific - Mediatek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8   v4l2_fourcc('M', 'B', 'B', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8   v4l2_fourcc('M', 'B', 'G', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8   v4l2_fourcc('M', 'B', 'g', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8   v4l2_fourcc('M', 'B', 'R', '8') /*  Packed  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10  v4l2_fourcc('M', 'B', 'B', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10  v4l2_fourcc('M', 'B', 'G', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10  v4l2_fourcc('M', 'B', 'g', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10  v4l2_fourcc('M', 'B', 'R', 'A') /*  Packed 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12  v4l2_fourcc('M', 'B', 'B', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12  v4l2_fourcc('M', 'B', 'G', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12  v4l2_fourcc('M', 'B', 'g', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12  v4l2_fourcc('M', 'B', 'R', 'C') /*  Packed 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14  v4l2_fourcc('M', 'B', 'B', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14  v4l2_fourcc('M', 'B', 'G', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14  v4l2_fourcc('M', 'B', 'g', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14  v4l2_fourcc('M', 'B', 'R', 'E') /*  Packed 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F  v4l2_fourcc('M', 'F', 'B', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F  v4l2_fourcc('M', 'F', 'G', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F  v4l2_fourcc('M', 'F', 'g', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F  v4l2_fourcc('M', 'F', 'R', '8') /*  Full-G  8-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F  v4l2_fourcc('M', 'F', 'B', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F  v4l2_fourcc('M', 'F', 'G', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F  v4l2_fourcc('M', 'F', 'g', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F  v4l2_fourcc('M', 'F', 'R', 'A') /*  Full-G 10-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F  v4l2_fourcc('M', 'F', 'B', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F  v4l2_fourcc('M', 'F', 'G', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F  v4l2_fourcc('M', 'F', 'g', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F  v4l2_fourcc('M', 'F', 'R', 'C') /*  Full-G 12-bit  */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F  v4l2_fourcc('M', 'F', 'B', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F  v4l2_fourcc('M', 'F', 'G', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F  v4l2_fourcc('M', 'F', 'g', 'E') /*  Full-G 14-bit  */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F  v4l2_fourcc('M', 'F', 'R', 'E') /*  Full-G 14-bit  */
+
 /* SDR formats - used only for Software Defined Radio devices */
 #define V4L2_SDR_FMT_CU8          v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
 #define V4L2_SDR_FMT_CU16LE       v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
@@ -749,6 +783,11 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
 #define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 #define V4L2_META_FMT_D4XX        v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /* AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /* Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /* Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /* ISP tuning parameters */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v5, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
  (?)
@ 2019-09-02  7:51     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11095113/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 11 files changed, 3369 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..89222e52bc7a 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
+
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..434dcd067b45
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..92948b4d69dd
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,634 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/remoteproc/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	struct device *dev = p1_dev->dev;
+	struct mtk_cam_dev_request *req;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+
+	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
+	if (req)
+		req->timestamp = ktime_get_boottime_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle if possible */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+				  (val & TG_CS_MASK) == TG_IDLE_ST,
+				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret) {
+		dev_err(dev, "failed to force suspend:%d\n", ret);
+		goto reenable_hw;
+	}
+
+	return 0;
+
+reenable_hw:
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return ret;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..452dc06110e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..16c742f57c40
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2081 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 ts_eof = ktime_get_boottime_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = ts_eof;
+		else
+			vb->timestamp = req->timestamp;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			spin_unlock_irqrestore(&cam->running_job_lock, flags);
+			return req;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+
+	return NULL;
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
+	vb2_request_queue(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	mtk_cam_dev_req_try_queue(cam);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..0a340a1e65ea
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @timestamp: Start of frame timestamp in ns
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	u64 timestamp;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC,v5, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11095113/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 11 files changed, 3369 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..89222e52bc7a 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
+
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..434dcd067b45
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..92948b4d69dd
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,634 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/remoteproc/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	struct device *dev = p1_dev->dev;
+	struct mtk_cam_dev_request *req;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+
+	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
+	if (req)
+		req->timestamp = ktime_get_boottime_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle if possible */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+				  (val & TG_CS_MASK) == TG_IDLE_ST,
+				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret) {
+		dev_err(dev, "failed to force suspend:%d\n", ret);
+		goto reenable_hw;
+	}
+
+	return 0;
+
+reenable_hw:
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return ret;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..452dc06110e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..16c742f57c40
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2081 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 ts_eof = ktime_get_boottime_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = ts_eof;
+		else
+			vb->timestamp = req->timestamp;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			spin_unlock_irqrestore(&cam->running_job_lock, flags);
+			return req;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+
+	return NULL;
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
+	vb2_request_queue(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	mtk_cam_dev_req_try_queue(cam);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..0a340a1e65ea
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @timestamp: Start of frame timestamp in ns
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	u64 timestamp;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [RFC, v5, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2019-09-02  7:51     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-09-02  7:51 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
This patch depends on "Add support for mt8183 SCP"[1].

[1] https://patchwork.kernel.org/cover/11095113/
---
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    2 +
 drivers/media/platform/mtk-isp/Kconfig        |   17 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  634 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2081 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 11 files changed, 3369 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8a19654b393a..672e3a74412b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -147,6 +147,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/mtk-isp/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7cbbd925124c..89222e52bc7a 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -92,6 +92,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_MDP)	+= mtk-mdp/
 
 obj-$(CONFIG_VIDEO_MEDIATEK_JPEG)	+= mtk-jpeg/
 
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1)	+= mtk-isp/isp_50/
+
 obj-$(CONFIG_VIDEO_QCOM_CAMSS)		+= qcom/camss/
 
 obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..434dcd067b45
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK || COMPILE_TEST
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..92948b4d69dd
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,634 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/remoteproc/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_IPI_SEND_TIMEOUT		50
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp_pdev, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp_pdev, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	struct device *dev = p1_dev->dev;
+	struct mtk_cam_dev_request *req;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+
+	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
+	if (req)
+		req->timestamp = ktime_get_boottime_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_warn(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	phandle rproc_phandle;
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp_pdev = scp_get_pdev(pdev);
+	if (!p1_dev->scp_pdev) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(dev->of_node, "mediatek,scp",
+				   &rproc_phandle);
+	if (ret) {
+		dev_err(dev, "failed to get rproc_phandle:%d\n", ret);
+		return -EINVAL;
+	}
+
+	p1_dev->rproc_handle = rproc_get_by_phandle(rproc_phandle);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	if (!p1_dev->rproc_handle) {
+		dev_err(dev, "failed to get rproc_handle\n");
+		return -EINVAL;
+	}
+	p1_dev->cam_dev.smem_dev = &p1_dev->scp_pdev->dev;
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr)
+		return -ENOMEM;
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  ptr, p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+
+	return ret;
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle if possible */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+				  (val & TG_CS_MASK) == TG_IDLE_ST,
+				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret) {
+		dev_err(dev, "failed to force suspend:%d\n", ret);
+		goto reenable_hw;
+	}
+
+	return 0;
+
+reenable_hw:
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return ret;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, 2 * MTK_ISP_STOP_HW_TIMEOUT);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	dma_free_coherent(&p1_dev->scp_pdev->dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	rproc_put(p1_dev->rproc_handle);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..452dc06110e2
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct platform_device *scp_pdev;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..16c742f57c40
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2081 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 ts_eof = ktime_get_boottime_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = ts_eof;
+		else
+			vb->timestamp = req->timestamp;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			spin_unlock_irqrestore(&cam->running_job_lock, flags);
+			return req;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+
+	return NULL;
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
+	vb2_request_queue(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	mtk_cam_dev_req_try_queue(cam);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..0a340a1e65ea
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @timestamp: Start of frame timestamp in ns
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	u64 timestamp;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [RFC, v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-09-02  7:51     ` Jungo Lin
  (?)
@ 2019-09-02 15:17       ` Rob Herring
  -1 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-09-02 15:17 UTC (permalink / raw)
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, jungo.lin, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

On Mon, 2 Sep 2019 15:51:31 +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 

Reviewed-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-09-02 15:17       ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-09-02 15:17 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

On Mon, 2 Sep 2019 15:51:31 +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 

Reviewed-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC, v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-09-02 15:17       ` Rob Herring
  0 siblings, 0 replies; 388+ messages in thread
From: Rob Herring @ 2019-09-02 15:17 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, jungo.lin, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, shik, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, sj.huang,
	tfiga, zwisler, ddavenport

On Mon, 2 Sep 2019 15:51:31 +0800, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
>  .../bindings/media/mediatek,camisp.txt        | 73 +++++++++++++++++++
>  1 file changed, 73 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 

Reviewed-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
  2019-05-13  8:46     ` Hans Verkuil
  (?)
@ 2019-10-02 10:55       ` Sakari Ailus
  -1 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 10:55 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, Jungo Lin, frederic.chen, seraph.huang,
	linux-media, devicetree, shik, yuzhao, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, sj.huang, tfiga, christie.yu, zwisler

Hi Jungo, Hans,

On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
...
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 
> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Jungo: what's the purpose of 

> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.
> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 
> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-10-02 10:55       ` Sakari Ailus
  0 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 10:55 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Jungo Lin, tfiga, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

Hi Jungo, Hans,

On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
...
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 
> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Jungo: what's the purpose of 

> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.
> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 
> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-10-02 10:55       ` Sakari Ailus
  0 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 10:55 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, Jungo Lin, frederic.chen, seraph.huang,
	linux-media, devicetree, shik, yuzhao, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, sj.huang, tfiga, christie.yu, zwisler

Hi Jungo, Hans,

On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> On 5/10/19 3:58 AM, Jungo Lin wrote:
...
> > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> 
> Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> that this is mediatek-specific. Same for the next control below.
> 
> > +	.name = "MTK CAM GET BIN INFO",
> > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.step = 1,
> > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> 
> Don't mix width and height. I recommend splitting this into two controls.
> 
> Sakari might have an opinion on this as well.
> 
> > +	},
> > +	{
> > +	.ops = &mtk_cam_dev_ctrl_ops,
> > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > +	.name = "MTK CAM RAW PATH",
> > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > +	.min = 0,
> > +	.max = 1,
> > +	.step = 1,
> > +	.def = 1,
> > +	},
> 
> RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> is 1, then it is 'processing raw'? If so, call it "Processing Raw".

Jungo: what's the purpose of 

> 
> Although you have to describe in the header or here what that means.
> 
> Private controls should be well documented.
> 
> > +};
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl)
> > +{
> > +	unsigned int i;
> > +
> > +	/* Initialized HW controls, allow V4L2_CID_MTK_CAM_MAX ctrls */
> > +	v4l2_ctrl_handler_init(hdl, V4L2_CID_MTK_CAM_MAX);
> > +	if (hdl->error) {
> > +		v4l2_ctrl_handler_free(hdl);
> > +		return hdl->error;
> > +	}
> > +
> > +	for (i = 0; i < ARRAY_SIZE(mtk_cam_controls); i++)
> > +		v4l2_ctrl_new_custom(hdl, &mtk_cam_controls[i], cam_dev);
> > +
> > +	dev_dbg(&cam_dev->pdev->dev, "%s done", __func__);
> > +	return 0;
> > +}
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > new file mode 100644
> > index 000000000000..74a6538c81ac
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ctrl.h
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Ryan Yu <ryan.yu@mediatek.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __MTK_CAM_CTRL_H__
> > +#define __MTK_CAM_CTRL_H__
> > +
> > +#include <media/v4l2-ctrls.h>
> > +
> > +#define V4L2_CID_MTK_CAM_PRIVATE_CAM  V4L2_CID_USER_MTK_CAM_BASE
> > +#define V4L2_CID_PRIVATE_GET_BIN_INFO \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 1)
> > +#define V4L2_CID_PRIVATE_RAW_PATH \
> > +	(V4L2_CID_MTK_CAM_PRIVATE_CAM + 2)
> 
> These last two defines can be on a single line.
> 
> They need to be documented in the header.
> 
> > +
> > +#define V4L2_CID_MTK_CAM_MAX	16
> > +
> > +int mtk_cam_ctrl_init(struct mtk_cam_dev *cam_dev,
> > +		      struct v4l2_ctrl_handler *hdl);
> > +
> > +#endif /* __MTK_CAM_CTRL_H__ */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 06479f2fb3ae..cbe8f5f7782b 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -192,6 +192,10 @@ enum v4l2_colorfx {
> >   * We reserve 16 controls for this driver. */
> >  #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
> >  
> > +/* The base for the mediatek ISP Pass 1 driver controls */
> > +/* We reserve 16 controls for this driver. */
> > +#define V4L2_CID_USER_MTK_CAM_BASE		(V4L2_CID_USER_BASE + 0x10c0)
> > +
> >  /* MPEG-class control IDs */
> >  /* The MPEG controls are applicable to all codec controls
> >   * and the 'MPEG' part of the define is historical */
> > 
> 
> Regards,
> 
> 	Hans

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
  2019-10-02 10:55       ` Sakari Ailus
  (?)
@ 2019-10-02 11:02         ` Sakari Ailus
  -1 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 11:02 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, Jungo Lin, frederic.chen, seraph.huang,
	linux-media, devicetree, shik, yuzhao, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, sj.huang, tfiga, christie.yu, zwisler

On Wed, Oct 02, 2019 at 01:55:59PM +0300, Sakari Ailus wrote:
> Hi Jungo, Hans,
> 
> On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> > On 5/10/19 3:58 AM, Jungo Lin wrote:
> ...
> > > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> > 
> > Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> > that this is mediatek-specific. Same for the next control below.
> > 
> > > +	.name = "MTK CAM GET BIN INFO",
> > > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.step = 1,
> > > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > 
> > Don't mix width and height. I recommend splitting this into two controls.
> > 
> > Sakari might have an opinion on this as well.
> > 
> > > +	},
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > > +	.name = "MTK CAM RAW PATH",
> > > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > > +	.min = 0,
> > > +	.max = 1,
> > > +	.step = 1,
> > > +	.def = 1,
> > > +	},
> > 
> > RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> > is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Jungo: what's the purpose of 

Please ignore the comment. I'll review a later version separately.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-10-02 11:02         ` Sakari Ailus
  0 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 11:02 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Jungo Lin, tfiga, laurent.pinchart+renesas, matthias.bgg,
	mchehab, linux-mediatek, linux-arm-kernel, linux-media,
	devicetree, srv_heupstream, Sean.Cheng, sj.huang, christie.yu,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, seraph.huang,
	ryan.yu, Rynn.Wu, yuzhao, zwisler, shik, suleiman

On Wed, Oct 02, 2019 at 01:55:59PM +0300, Sakari Ailus wrote:
> Hi Jungo, Hans,
> 
> On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> > On 5/10/19 3:58 AM, Jungo Lin wrote:
> ...
> > > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> > 
> > Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> > that this is mediatek-specific. Same for the next control below.
> > 
> > > +	.name = "MTK CAM GET BIN INFO",
> > > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.step = 1,
> > > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > 
> > Don't mix width and height. I recommend splitting this into two controls.
> > 
> > Sakari might have an opinion on this as well.
> > 
> > > +	},
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > > +	.name = "MTK CAM RAW PATH",
> > > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > > +	.min = 0,
> > > +	.max = 1,
> > > +	.step = 1,
> > > +	.def = 1,
> > > +	},
> > 
> > RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> > is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Jungo: what's the purpose of 

Please ignore the comment. I'll review a later version separately.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control
@ 2019-10-02 11:02         ` Sakari Ailus
  0 siblings, 0 replies; 388+ messages in thread
From: Sakari Ailus @ 2019-10-02 11:02 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart+renesas, Rynn.Wu,
	suleiman, Jerry-ch.Chen, Jungo Lin, frederic.chen, seraph.huang,
	linux-media, devicetree, shik, yuzhao, linux-mediatek,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, sj.huang, tfiga, christie.yu, zwisler

On Wed, Oct 02, 2019 at 01:55:59PM +0300, Sakari Ailus wrote:
> Hi Jungo, Hans,
> 
> On Mon, May 13, 2019 at 10:46:46AM +0200, Hans Verkuil wrote:
> > On 5/10/19 3:58 AM, Jungo Lin wrote:
> ...
> > > +struct v4l2_ctrl_config mtk_cam_controls[] = {
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_GET_BIN_INFO,
> > 
> > Don't use "PRIVATE" in the name. I'd replace that with MTK to indicate
> > that this is mediatek-specific. Same for the next control below.
> > 
> > > +	.name = "MTK CAM GET BIN INFO",
> > > +	.type = V4L2_CTRL_TYPE_INTEGER,
> > > +	.min = (IMG_MIN_WIDTH << 16) | IMG_MIN_HEIGHT,
> > > +	.max = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.step = 1,
> > > +	.def = (IMG_MAX_WIDTH << 16) | IMG_MAX_HEIGHT,
> > > +	.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE,
> > 
> > Don't mix width and height. I recommend splitting this into two controls.
> > 
> > Sakari might have an opinion on this as well.
> > 
> > > +	},
> > > +	{
> > > +	.ops = &mtk_cam_dev_ctrl_ops,
> > > +	.id = V4L2_CID_PRIVATE_RAW_PATH,
> > > +	.name = "MTK CAM RAW PATH",
> > > +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> > > +	.min = 0,
> > > +	.max = 1,
> > > +	.step = 1,
> > > +	.def = 1,
> > > +	},
> > 
> > RAW_PATH is a very vague name. If it is 0, then it is pure raw, and if it
> > is 1, then it is 'processing raw'? If so, call it "Processing Raw".
> 
> Jungo: what's the purpose of 

Please ignore the comment. I'll review a later version separately.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
       [not found] <Jungo Lin <jungo.lin@mediatek.com>
@ 2019-12-19  5:49   ` Jungo Lin
  2019-04-03  1:44   ` Jungo Lin
                     ` (15 subsequent siblings)
  16 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Hello,

This patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
* Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
  Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP
  communication to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor, sen-inf sub-devices and len device
which are implemented in below patch[1][2][3][4]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
patch sets.

/usr/bin/media-ctl -p -d /dev/media2

Media controller API version 4.19.89

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.89

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video8
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
	pad0: Sink
		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
		<- "ov8856 2-0010":0 [ENABLED]
	pad1: Sink
		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov02a10 4-003d":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov8856 2-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
		-> "1a040000.seninf":0 [ENABLED]

- entity 73: dw9768 2-000c (0 pad, 0 link)
             type V4L2 subdev subtype Lens flags 0
             device node name /dev/v4l-subdev3

- entity 74: ov02a10 4-003d (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev4
	pad0: Source
		[fmt:SRGGB10_1X10/1600x1200 field:none]
		-> "1a040000.seninf":1 []

===========
= history =
===========

version 6:
 - Add port node description in the dt-binding document and device tree
   by Tomasz Figa.
 - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
   version.
 - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
 - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
 - Correct auto suspend timer value for suspend/resume issue.
 - Increase IPI guard timer to 1 second to avoid false alarm command
   timeout event.
 - Fix KE due to no sen-inf sub-device.

Todo:
 - vb2_ops's buf_request_complete callback function implementation.
 - Add rst documents for Mediatek meta formats.
 - New meta buffer structure design & re-factoring.

version 5:
 - Fixed Rob's comment on dt-binding format
 - Fix Tomasz's comment in mtk_isp_pm_suspend function
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
   and new timestamp type in driver
 - Fix buffer en-queue timing issue in v4
 - Remove default link_notify callback function in mtk_cam_media_ops

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Align and pack IPI command structures for EC ROM size shrink

version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
   patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend Mediatek proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI command structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
   declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height
   in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch set
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/11145845/

[2]. media: ov8856: Add YAML binding and sensor mode support
https://patchwork.kernel.org/cover/11220785/

[3]. media: i2c: Add support for OV02A10 sensor
https://patchwork.kernel.org/cover/11284779/

[4]. media: i2c: add support for DW9768 VCM driver
https://patchwork.kernel.org/cover/11132299/

[5]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11239065/

[6]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/11112765/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158

Note 1.
This testing depends on the above seninf, sensors and len patches[1][2][3][4].

Note 2.
For failed test csaes in video2~8, it is caused by new V4L2 timestamp
called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.

Note 3.
The current some failure items are related to Mediatek sensors/len driver [2][3][3]

/usr/bin/v4l2-compliance -m /dev/media2

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           :
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composiv4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media2:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media2 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video2:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x8c200000
		Metadata Output
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Metadata Output
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video2 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video3:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video4:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video4 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video5:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video5 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video6:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video6 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video7:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video7 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video8:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video8 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000050
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev0 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000052
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev1 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000054
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov8856 2-0010
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev2 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000056
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : dw9768 2-000c
	Function         : Lens Controller

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev3 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000058
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000004a (74)
	Name             : ov02a10 4-003d
	Function         : Camera Sensor
	Pad 0x0100004b   : 0: Source
	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
		fail: v4l2-test-controls.cpp(362): returned control value out of range
		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
	test VIDIOC_G/S_CTRL: FAIL
		fail: v4l2-test-controls.cpp(549): returned control value out of range
		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0


Jungo Lin (5):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: videodev2.h: Add new boottime timestamp type
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   83 +
 Documentation/media/uapi/v4l/buffer.rst       |   11 +-
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   20 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   41 +
 24 files changed, 4226 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2019-12-19  5:49   ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hello,

This patch series adding the driver for Pass 1 (P1) unit in
Mediatek's camera ISP system on mt8183 SoC, which will be used in
camera features of CrOS.

Pass 1 unit processes image signal from sensor devices and accepts the
tuning parameters to adjust the image quality. It performs optical
black correction, defect pixel correction, W/IR imbalance correction
and lens shading correction for RAW processing.

The driver is implemented with V4L2 and media controller framework so
we have the following entities to describe the ISP pass 1 path.

(The current metadata interface used in meta input and partial meta
nodes is only a temporary solution to kick off the driver development
and is not ready to be reviewed yet.)

1. meta input (output video device): connect to ISP P1 sub device.
It accepts the tuning buffer from user.

2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
main stream and packed out video devices. When processing an image,
Pass 1 hardware supports multiple output images with different sizes
and formats so it needs two capture video devices ("main stream" and
"packed out") to return the image data to the user.

3. main stream (capture video device): return the processed image data
which is used in capture scenario.

4. packed out (capture video device): return the processed image data
which is used in preview scenario.

5. partial meta 0 (capture video device): return the AE/AWB statistics.

6. partial meta 1 (capture video device): return the AF statistics.

7. partial meta 2 (capture video device): return the local contrast
   enhanced statistics.

8. partial meta 3 (capture video device): return the local motion
   vector statistics.

The overall patches of the series is:

* Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
* Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
* Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
* Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
  Moreover, implement standard V4L2 video driver that utilizes
  V4L2 and media framework APIs. Communicate with co-process via SCP
  communication to compose ISP registers in the firmware.

Here is ISP P1 media topology:
It is included the main/sub sensor, sen-inf sub-devices and len device
which are implemented in below patch[1][2][3][4]:

For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
patch sets.

/usr/bin/media-ctl -p -d /dev/media2

Media controller API version 4.19.89

Media device information
------------------------
driver          mtk-cam-p1
model           mtk-cam-p1
serial          
bus info        platform:1a000000.camisp
hw revision     0x0
driver version  4.19.89

Device topology
- entity 1: mtk-cam-p1 (12 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "mtk-cam-p1 meta input":0 []
	pad1: Source
		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "mtk-cam-p1 packed out":0 []
	pad3: Source
		-> "mtk-cam-p1 partial meta 0":0 []
	pad4: Source
		-> "mtk-cam-p1 partial meta 1":0 []
	pad5: Source
		-> "mtk-cam-p1 partial meta 2":0 []
	pad6: Source
		-> "mtk-cam-p1 partial meta 3":0 []
	pad7: Source
	pad8: Source
	pad9: Source
	pad10: Source
	pad11: Sink
		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]

- entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Source
		-> "mtk-cam-p1":0 []

- entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]

- entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Sink
		<- "mtk-cam-p1":2 []

- entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Sink
		<- "mtk-cam-p1":3 []

- entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Sink
		<- "mtk-cam-p1":4 []

- entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
	pad0: Sink
		<- "mtk-cam-p1":5 []

- entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video8
	pad0: Sink
		<- "mtk-cam-p1":6 []

- entity 56: 1a040000.seninf (12 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
	pad0: Sink
		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
		<- "ov8856 2-0010":0 [ENABLED]
	pad1: Sink
		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
		<- "ov02a10 4-003d":0 []
	pad2: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad3: Sink
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad4: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
	pad5: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad6: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad7: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad8: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad9: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad10: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
	pad11: Source
		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]

- entity 69: ov8856 2-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
		-> "1a040000.seninf":0 [ENABLED]

- entity 73: dw9768 2-000c (0 pad, 0 link)
             type V4L2 subdev subtype Lens flags 0
             device node name /dev/v4l-subdev3

- entity 74: ov02a10 4-003d (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev4
	pad0: Source
		[fmt:SRGGB10_1X10/1600x1200 field:none]
		-> "1a040000.seninf":1 []

===========
= history =
===========

version 6:
 - Add port node description in the dt-binding document and device tree
   by Tomasz Figa.
 - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
   version.
 - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
 - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
 - Correct auto suspend timer value for suspend/resume issue.
 - Increase IPI guard timer to 1 second to avoid false alarm command
   timeout event.
 - Fix KE due to no sen-inf sub-device.

Todo:
 - vb2_ops's buf_request_complete callback function implementation.
 - Add rst documents for Mediatek meta formats.
 - New meta buffer structure design & re-factoring.

version 5:
 - Fixed Rob's comment on dt-binding format
 - Fix Tomasz's comment in mtk_isp_pm_suspend function
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
   and new timestamp type in driver
 - Fix buffer en-queue timing issue in v4
 - Remove default link_notify callback function in mtk_cam_media_ops

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Align and pack IPI command structures for EC ROM size shrink

version 4:
 - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
   patch[4]
 - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
 - Extend Mediatek proprietary image formats to support bayer order
 - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices

Todo:
 - vb2_ops's buf_request_complete callback function implementation
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring
 - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
 - Align and pack IPI command structures for EC ROM size shrink

version 3:
 - Remove ISP Pass 1 reserved memory device node and change to use SCP's
   reserved memory region. (Rob Herring)
 - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
 - Revise ISP Pass1 Kconfig
 - Add rst documents for Mediatek image formats (Hans Verkuil)
 - Fix kernel warning messages when running v4l2_compliance test
 - Move AFO buffer enqueue & de-queue from request API to non-request
 - mtk_cam-ctrl.h/mtk_cam-ctrl.c
   Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
   declaration (Hans Verkuil)
   Split GET_BIN_INFO control into two controls to get width & height
   in-dependently (Hans Verkuil)
 - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
   Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
   Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Fix Drew's comments which are addressed in v2 patch
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
 - mtk_cam-scp.h / mtk_cam-scp.c
   Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
   Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
   Fix ISP de-initialize timing KE issue
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Get the reserved shared memory via SCP driver (Tomasz Figa)

Todo:
 - Add rst documents for Mediatek meta formats
 - New meta buffer structure design & re-factoring

version 2:
 - Add 3A enhancement feature which includes:
   Separates 3A pipeline out of frame basis to improve
   AE/AWB (exposure and white balance) performance.
   Add 2 SCP sub-commands for 3A meta buffers.
 - Add new child device to manage P1 shared memory between P1 HW unit
   and co-processor.
 - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
 - Revised document wording for dt-bindings documents & dts information.
 - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
   source codes to mtk_cam-dev.h & mtk_cam-dev.c.
 - mtk_cam-dev.h / mtk_cam-dev.c
   Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
   or add comments.
   Revised buffer size for LMVO & LCSO.
   Fix pixel format utility function.
   Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
 - mtk_cam-v4l2-util.c
   Refactoring V4L2 async mechanism with seninf driver only
   Refactoring CIO (Connection IO) implementation with active sensor
   Revised stream on function for 3A enhancement feature
   Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
 - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
   Add meta buffer index register definitions
   Add meta DMA configuration function.
   Separate with frame-base and non-frame-base en-queue/de-queue functions
   Add isp_setup_scp_rproc function to get RPC handle
   Add mtk_cam_reserved_memory_init for shared memory management
 - mtk_cam-scp.h / mtk_cam-scp.c
   Add new meta strictures for 3A enhancement feature
   Add new IPI command utility function for 3A enhancement feature
   Enhance isp_composer_dma_sg_init function flow
   Shorten overall IPI command structure size
   Remove scp_state state checking
   Improve code readability
 - mtk_cam-smem.h / mtk_cam-smem-dev.c
   Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
   Handling P1 driver 's reserved memory & allocate DMA buffers based on this
   memory region.

TODOs:
 - 3A enhancement feature bug fixing

version 1:
 - Revised driver sources based on Tomasz's comments including
   part1/2/3/4 in RFC V0 patch.
 - Remove DMA cache mechanism.
   Support two new video devices (LCSO/LMVO) for advance camera
   features.
 - Fixed v4l2-compliance test failure items.
 - Add private controls for Mediatek camera middle-ware.
 - Replace VPU driver's APIs with new SCP driver interface for
   co-processor communication.
 - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
   commands RX handling.
 - Fix internal bugs.

TODOs:
 - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
   for shared memory management.
 - Revised file names.
 - Support non frame-sync AFO/AAO DMA buffers

version 0:
- Initial submission

==================
 Dependent patch set
==================

Camera ISP P1 driver depends on seninf driver, SCP driver.
The patches are listed as following:

[1]. media: support Mediatek sensor interface driver
https://patchwork.kernel.org/cover/11145845/

[2]. media: ov8856: Add YAML binding and sensor mode support
https://patchwork.kernel.org/cover/11220785/

[3]. media: i2c: Add support for OV02A10 sensor
https://patchwork.kernel.org/cover/11284779/

[4]. media: i2c: add support for DW9768 VCM driver
https://patchwork.kernel.org/cover/11132299/

[5]. Add support for mt8183 SCP
https://patchwork.kernel.org/cover/11239065/

[6]. MT8183 IOMMU SUPPORT
https://patchwork.kernel.org/cover/11112765/

==================
 Compliance test
==================

The v4l2-compliance is built with the below lastest patch.
https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158

Note 1.
This testing depends on the above seninf, sensors and len patches[1][2][3][4].

Note 2.
For failed test csaes in video2~8, it is caused by new V4L2 timestamp
called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.

Note 3.
The current some failure items are related to Mediatek sensors/len driver [2][3][3]

/usr/bin/v4l2-compliance -m /dev/media2

v4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           :
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media1 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video25:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.67
	Capabilities     : 0x8c200000
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.67
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.67
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video25 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composiv4l2-compliance SHA: not available, 32 bits

Compliance test for mtk-cam-p1 device /dev/media2:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK

Allow for multiple opens:
	test second /dev/media2 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video2:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x8c200000
		Metadata Output
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x0c200000
		Metadata Output
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000010
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000e (14)
	Name             : mtk-cam-p1 meta input
	Function         : V4L2 I/O
	Pad 0x0100000f   : 0: Source
	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video2 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video3:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000016
	Type             : V4L Video
Entity Info:
	ID               : 0x00000014 (20)
	Name             : mtk-cam-p1 main stream
	Function         : V4L2 I/O
	Pad 0x01000015   : 0: Sink
	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video4:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04201000
		Video Capture Multiplanar
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x0300001c
	Type             : V4L Video
Entity Info:
	ID               : 0x0000001a (26)
	Name             : mtk-cam-p1 packed out
	Function         : V4L2 I/O
	Pad 0x0100001b   : 0: Sink
	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video4 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video5:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000022
	Type             : V4L Video
Entity Info:
	ID               : 0x00000020 (32)
	Name             : mtk-cam-p1 partial meta 0
	Function         : V4L2 I/O
	Pad 0x01000021   : 0: Sink
	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video5 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video6:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000028
	Type             : V4L Video
Entity Info:
	ID               : 0x00000026 (38)
	Name             : mtk-cam-p1 partial meta 1
	Function         : V4L2 I/O
	Pad 0x01000027   : 0: Sink
	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video6 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video7:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x0300002e
	Type             : V4L Video
Entity Info:
	ID               : 0x0000002c (44)
	Name             : mtk-cam-p1 partial meta 2
	Function         : V4L2 I/O
	Pad 0x0100002d   : 0: Sink
	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video7 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/video8:

Driver Info:
	Driver name      : mtk-cam-p1
	Card type        : mtk-cam-p1
	Bus info         : platform:1a000000.camisp
	Driver version   : 4.19.89
	Capabilities     : 0x84a00000
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000034
	Type             : V4L Video
Entity Info:
	ID               : 0x00000032 (50)
	Name             : mtk-cam-p1 partial meta 3
	Function         : V4L2 I/O
	Pad 0x01000033   : 0: Sink
	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video8 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK

Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000050
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : mtk-cam-p1
	Function         : Video Pixel Formatter
	Pad 0x01000002   : 0: Sink
	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
	Pad 0x01000003   : 1: Source
	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
	Pad 0x01000005   : 3: Source
	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
	Pad 0x01000006   : 4: Source
	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
	Pad 0x01000007   : 5: Source
	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
	Pad 0x01000008   : 6: Source
	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
	Pad 0x01000009   : 7: Source
	Pad 0x0100000a   : 8: Source
	Pad 0x0100000b   : 9: Source
	Pad 0x0100000c   : 10: Source
	Pad 0x0100000d   : 11: Sink
	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev0 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000052
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000038 (56)
	Name             : 1a040000.seninf
	Function         : Video Interface Bridge
	Pad 0x01000039   : 0: Sink
	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
	Pad 0x0100003a   : 1: Sink
	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
	Pad 0x0100003b   : 2: Sink
	Pad 0x0100003c   : 3: Sink
	Pad 0x0100003d   : 4: Source
	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
	Pad 0x0100003e   : 5: Source
	Pad 0x0100003f   : 6: Source
	Pad 0x01000040   : 7: Source
	Pad 0x01000041   : 8: Source
	Pad 0x01000042   : 9: Source
	Pad 0x01000043   : 10: Source
	Pad 0x01000044   : 11: Source

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev1 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Sink Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 1):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 2):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Sink Pad 3):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 4):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 5):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 6):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 7):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 8):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 9):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 10):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Sub-Device ioctls (Source Pad 11):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000054
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000045 (69)
	Name             : ov8856 2-0010
	Function         : Camera Sensor
	Pad 0x01000046   : 0: Source
	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev2 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 11 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000056
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000049 (73)
	Name             : dw9768 2-000c
	Function         : Lens Controller

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev3 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:

Media Driver Info:
	Driver name      : mtk-cam-p1
	Model            : mtk-cam-p1
	Serial           : 
	Bus info         : platform:1a000000.camisp
	Media version    : 4.19.89
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.19.89
Interface Info:
	ID               : 0x03000058
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x0000004a (74)
	Name             : ov02a10 4-003d
	Function         : Camera Sensor
	Pad 0x0100004b   : 0: Source
	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK

Allow for multiple opens:
	test second /dev/v4l-subdev4 open: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device ioctls (Source Pad 0):
	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Try VIDIOC_SUBDEV_G/S_FMT: OK
	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
	test Active VIDIOC_SUBDEV_G/S_FMT: OK
	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
		fail: v4l2-test-controls.cpp(362): returned control value out of range
		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
	test VIDIOC_G/S_CTRL: FAIL
		fail: v4l2-test-controls.cpp(549): returned control value out of range
		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 10 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0

Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0


Jungo Lin (5):
  media: dt-bindings: mt8183: Added camera ISP Pass 1
  dts: arm64: mt8183: Add ISP Pass 1 nodes
  media: videodev2.h: Add new boottime timestamp type
  media: platform: Add Mediatek ISP P1 image & meta formats
  media: platform: Add Mediatek ISP P1 V4L2 device driver

 .../bindings/media/mediatek,camisp.txt        |   83 +
 Documentation/media/uapi/v4l/buffer.rst       |   11 +-
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
 arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/mtk-isp/Kconfig        |   20 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
 include/uapi/linux/videodev2.h                |   41 +
 24 files changed, 4226 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-12-19  5:49   ` Jungo Lin
  (?)
@ 2019-12-19  5:49     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Add port node description in the dt-binding document.
---
 .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
 1 file changed, 83 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..a85f37c0b87d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,83 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp: The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- port: child port node corresponding to the data input, in accordance with
+  the video interface bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
+  node must contain at least one endpoint.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+
+		port {
+			camisp_endpoint: endpoint {
+				remote-endpoint = <&seninf_camisp_endpoint>;
+			};
+		};
+	};
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Add port node description in the dt-binding document.
---
 .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
 1 file changed, 83 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..a85f37c0b87d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,83 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp: The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- port: child port node corresponding to the data input, in accordance with
+  the video interface bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
+  node must contain at least one endpoint.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+
+		port {
+			camisp_endpoint: endpoint {
+				remote-endpoint = <&seninf_camisp_endpoint>;
+			};
+		};
+	};
-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

This patch adds DT binding document for the Pass 1 (P1) unit
in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
data out from the sensor interface, applies ISP image effects
from tuning data and outputs the image data or statistics data to DRAM.

Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Add port node description in the dt-binding document.
---
 .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
 1 file changed, 83 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt

diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
new file mode 100644
index 000000000000..a85f37c0b87d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
@@ -0,0 +1,83 @@
+* Mediatek Image Signal Processor Pass 1 (ISP P1)
+
+The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
+from the sensor interface, applies ISP effects from tuning data and outputs
+the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
+the ability to output two different resolutions frames at the same time to
+increase the performance of the camera application.
+
+Required properties:
+- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
+- reg: Physical base address of the camera function block register and
+  length of memory mapped region. Must contain an entry for each entry
+  in reg-names.
+- reg-names: Must include the following entries:
+  "cam_sys": Camera base function block
+  "cam_uni": Camera UNI function block
+  "cam_a": Camera ISP P1 hardware unit A
+  "cam_b": Camera ISP P1 hardware unit B
+  "cam_c": Camera ISP P1 hardware unit C
+- interrupts: Must contain an entry for each entry in interrupt-names.
+- interrupt-names : Must include the following entries:
+  "cam_uni": Camera UNI interrupt
+  "cam_a": Camera unit A interrupt
+  "cam_b": Camera unit B interrupt
+  "cam_c": Camera unit C interrupt
+- iommus: Shall point to the respective IOMMU block with master port
+  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+  for details.
+- clocks: A list of phandle and clock specifier pairs as listed
+  in clock-names property, see
+  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
+- mediatek,larb: Must contain the local arbiters in the current SoCs, see
+  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
+  for details.
+- power-domains: a phandle to the power domain, see
+  Documentation/devicetree/bindings/power/power_domain.txt for details.
+- mediatek,scp: The node of system control processor (SCP), see
+  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
+- port: child port node corresponding to the data input, in accordance with
+  the video interface bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
+  node must contain at least one endpoint.
+
+Example:
+SoC specific DT entry:
+
+	camisp: camisp@1a000000 {
+		compatible = "mediatek,mt8183-camisp";
+		reg = <0 0x1a000000 0 0x1000>,
+				<0 0x1a003000 0 0x1000>,
+				<0 0x1a004000 0 0x2000>,
+				<0 0x1a006000 0 0x2000>,
+				<0 0x1a008000 0 0x2000>;
+		reg-names = "cam_sys",
+				"cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+		interrupt-names = "cam_uni",
+				"cam_a",
+				"cam_b",
+				"cam_c";
+		iommus = <&iommu M4U_PORT_CAM_IMGO>;
+		clocks = <&camsys CLK_CAM_CAM>,
+				<&camsys CLK_CAM_CAMTG>;
+		clock-names = "camsys_cam_cgpdn",
+				"camsys_camtg_cgpdn";
+		mediatek,larb = <&larb3>,
+				<&larb6>;
+		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+		mediatek,scp = <&scp>;
+
+		port {
+			camisp_endpoint: endpoint {
+				remote-endpoint = <&seninf_camisp_endpoint>;
+			};
+		};
+	};
-- 
2.18.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
  2019-12-19  5:49   ` Jungo Lin
  (?)
@ 2019-12-19  5:49     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
---
Changes from v6:
 - Add port node description in the device tree by Tomasz Figa.
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 38 ++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 10b32471bc7b..7a5349371b9f 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -641,5 +641,43 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+
+			port {
+				camisp_endpoint: endpoint {
+					remote-endpoint = <&seninf_camisp_endpoint>;
+				};
+			};
+		};
+		};
 	};
 };
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
---
Changes from v6:
 - Add port node description in the device tree by Tomasz Figa.
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 38 ++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 10b32471bc7b..7a5349371b9f 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -641,5 +641,43 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+
+			port {
+				camisp_endpoint: endpoint {
+					remote-endpoint = <&seninf_camisp_endpoint>;
+				};
+			};
+		};
+		};
 	};
 };
-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add nodes for Pass 1 unit of Mediatek's camera ISP system.
Pass 1 unit embedded in Mediatek SoCs, works with the
co-processor to process image signal from the image sensor
and output RAW image data.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
---
Changes from v6:
 - Add port node description in the device tree by Tomasz Figa.
---
 arch/arm64/boot/dts/mediatek/mt8183.dtsi | 38 ++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index 10b32471bc7b..7a5349371b9f 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -641,5 +641,43 @@
 			reg = <0 0x1a000000 0 0x1000>;
 			#clock-cells = <1>;
 		};
+
+		camisp: camisp@1a000000 {
+			compatible = "mediatek,mt8183-camisp";
+			reg = <0 0x1a000000 0 0x1000>,
+					<0 0x1a003000 0 0x1000>,
+					<0 0x1a004000 0 0x2000>,
+					<0 0x1a006000 0 0x2000>,
+					<0 0x1a008000 0 0x2000>;
+			reg-names = "cam_sys",
+					"cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
+					<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
+			interrupt-names = "cam_uni",
+					"cam_a",
+					"cam_b",
+					"cam_c";
+			iommus = <&iommu M4U_PORT_CAM_IMGO>;
+			clocks = <&camsys CLK_CAM_CAM>,
+					<&camsys CLK_CAM_CAMTG>;
+			clock-names = "camsys_cam_cgpdn",
+					"camsys_camtg_cgpdn";
+			mediatek,larb = <&larb3>,
+					<&larb6>;
+			power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
+			mediatek,scp = <&scp>;
+
+			port {
+				camisp_endpoint: endpoint {
+					remote-endpoint = <&seninf_camisp_endpoint>;
+				};
+			};
+		};
+		};
 	};
 };
-- 
2.18.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
  2019-12-19  5:49   ` Jungo Lin
  (?)
@ 2019-12-19  5:49     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - No change.
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 9149b57728e5..f45bfce7fddd 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -662,13 +662,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 04481c717fee..74ef9472e702 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - No change.
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 9149b57728e5..f45bfce7fddd 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -662,13 +662,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 04481c717fee..74ef9472e702 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

For Camera AR(Augmented Reality) application requires camera timestamps
to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
sources.

The boottime timestamp is identical to monotonic timestamp,
except it also includes any time that the system is suspended.

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - No change.
---
 Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
 include/uapi/linux/videodev2.h          |  2 ++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 9149b57728e5..f45bfce7fddd 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -662,13 +662,22 @@ Buffer Flags
       - 0x00002000
       - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
 	clock. To access the same clock outside V4L2, use
-	:c:func:`clock_gettime`.
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
       - 0x00004000
       - The CAPTURE buffer timestamp has been taken from the corresponding
 	OUTPUT buffer. This flag applies only to mem2mem devices.
+    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
+
+      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
+      - 0x00008000
+      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
+	clock. To access the same clock outside V4L2, use
+	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
+	Identical to CLOCK_MONOTONIC, except it also includes any time that
+	the system is suspended.
     * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
 
       - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 04481c717fee..74ef9472e702 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
+
 /* Timestamp sources. */
 #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
 #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
-- 
2.18.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-12-19  5:49   ` Jungo Lin
  (?)
@ 2019-12-19  5:49     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Remove RGB format definitions in pixfmt-rgb.rst for kernel
   v5.5-rc1 version.
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 8 files changed, 638 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Remove RGB format definitions in pixfmt-rgb.rst for kernel
   v5.5-rc1 version.
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 8 files changed, 638 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, jungo.lin, sj.huang,
	yuzhao, linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Add packed/full-g bayer formats with 8/10/12/14 bit
for image output. Add Pass 1 (P1) specific meta formats for
parameter processing and 3A/other statistics.

(The current metadata format used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
---
Changes from v6:
 - Remove RGB format definitions in pixfmt-rgb.rst for kernel
   v5.5-rc1 version.
---
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
 .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
 8 files changed, 638 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
new file mode 100644
index 000000000000..534edb4f0fd4
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
@@ -0,0 +1,65 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10:
+.. _v4l2-pix-fmt-mtisp-sgbrg10:
+.. _v4l2-pix-fmt-mtisp-sgrbg10:
+.. _v4l2-pix-fmt-mtisp-srggb10:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
+*******************************
+
+10-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 10 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+    * - start + 2:
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - G\ :sub:`03high bits 9--2`
+    * - start + 6:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+    * - start + 8:
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 10:
+      - R\ :sub:`13high bits 9--2`
+    * - start + 12:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+    * - start + 14:
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 16:
+      - G\ :sub:`23high bits 9--2`
+    * - start + 18:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+    * - start + 20:
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 22:
+      - R\ :sub:`33high bits 9--2` (bits 7--0)
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
new file mode 100644
index 000000000000..7be527711602
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
@@ -0,0 +1,90 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr10f:
+.. _v4l2-pix-fmt-mtisp-sgbrg10f:
+.. _v4l2-pix-fmt-mtisp-sgrbg10f:
+.. _v4l2-pix-fmt-mtisp-srggb10f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
+*******************************
+
+10-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 10 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
+    * - start + 4:
+      - B\ :sub:`03high bits 9--2`
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`05high bits 3--0`
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
+    * - start + 12:
+      - G\ :sub:`13high bits 9--2`
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`15high bits 3--0`
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
+      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
+    * - start + 20:
+      - B\ :sub:`23high bits 9--2`
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
+      - G\ :sub:`25high bits 3--0`
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
+      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
+    * - start + 28:
+      - G\ :sub:`33high bits 9--2`
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
+      - FG\ :sub:`35high bits 3--0`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
new file mode 100644
index 000000000000..cc888aac42c2
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
@@ -0,0 +1,61 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12:
+.. _v4l2-pix-fmt-mtisp-sgbrg12:
+.. _v4l2-pix-fmt-mtisp-sgrbg12:
+.. _v4l2-pix-fmt-mtisp-srggb12:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
+*******************************
+
+12-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 12 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00lowbits 7--0`
+      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`01highbits 7--0`
+      - B\ :sub:`02lowbits 7--0`
+      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`03highbits 7--0`
+    * - start + 6:
+      - G\ :sub:`10lowbits 7--0`
+      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`11highbits 7--0`
+      - G\ :sub:`12lowbits 7--0`
+      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`13highbits 7--0`
+    * - start + 12:
+      - B\ :sub:`20lowbits 7--0`
+      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`21highbits 7--0`
+      - B\ :sub:`22lowbits 7--0`
+      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
+      - G\ :sub:`23highbits 7--0`
+    * - start + 18:
+      - G\ :sub:`30lowbits 7--0`
+      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`31highbits 7--0`
+      - G\ :sub:`32lowbits 7--0`
+      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
+      - R\ :sub:`33highbits 7--0`
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
new file mode 100644
index 000000000000..c063de9f9ad8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr12f:
+.. _v4l2-pix-fmt-mtisp-sgbrg12f:
+.. _v4l2-pix-fmt-mtisp-sgrbg12f:
+.. _v4l2-pix-fmt-mtisp-srggb12f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
+*******************************
+
+12-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 12 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
+    * - start + 2:
+      - FG\ :sub:`01high bits 7--0`
+      - G\ :sub:`02low bits 7--0`
+    * - start + 4:
+      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`03high bits 7--0`
+    * - start + 6:
+      - FG\ :sub:`04low bits 7--0`
+      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
+    * - start + 8:
+      - G\ :sub:`05high bits 7--0`
+      -
+    * - start + 10:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
+    * - start + 12:
+      - R\ :sub:`11high bits 7--0`
+      - FG\ :sub:`12low bits 7--0`
+    * - start + 14:
+      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`13high bits 7--0`
+    * - start + 16:
+      - R\ :sub:`14low bits 7--0`
+      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
+    * - start + 18:
+      - FG\ :sub:`15high bits 7--0`
+      -
+    * - start + 20:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
+    * - start + 22:
+      - FG\ :sub:`21high bits 7--0`
+      - G\ :sub:`22low bits 7--0`
+    * - start + 24:
+      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
+      - B\ :sub:`23high bits 7--0`
+    * - start + 26:
+      - FG\ :sub:`24low bits 7--0`
+      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`25high bits 7--0`
+      -
+    * - start + 30:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
+    * - start + 32:
+      - R\ :sub:`31high bits 7--0`
+      - FG\ :sub:`32low bits 7--0`
+    * - start + 34:
+      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
+      - G\ :sub:`33high bits 7--0`
+    * - start + 36:
+      - R\ :sub:`34low bits 7--0`
+      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
+    * - start + 38:
+      - FG\ :sub:`35high bits 7--0`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
new file mode 100644
index 000000000000..39ea9882a792
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
@@ -0,0 +1,73 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14:
+.. _v4l2-pix-fmt-mtisp-sgbrg14:
+.. _v4l2-pix-fmt-mtisp-sgrbg14:
+.. _v4l2-pix-fmt-mtisp-srggb14:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
+*******************************
+
+14-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 14 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`01low bits 9--2`\
+      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - B\ :sub:`02low bits 11--4`\
+      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`03high bits 13--6`\
+      -
+    * - start + 8:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`\
+      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 12:
+      - G\ :sub:`12low bits 11--4`\
+      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`13high bits 13--6`\
+      -
+    * - start + 16:
+      - B\ :sub:`20low bits 7--0`
+      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`21low bits 9--2`\
+      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 20:
+      - B\ :sub:`22low bits 11--4`\
+      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`23high bits 13--6`\
+      -
+    * - start + 24:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`\
+      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`32low bits 11--4`\
+      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - R\ :sub:`33high bits 13--6`\
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
new file mode 100644
index 000000000000..010b1c190c60
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
@@ -0,0 +1,110 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr14f:
+.. _v4l2-pix-fmt-mtisp-sgbrg14f:
+.. _v4l2-pix-fmt-mtisp-sgrbg14f:
+.. _v4l2-pix-fmt-mtisp-srggb14f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
+*******************************
+
+14-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 14 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00low bits 7--0`
+      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`01low bits 9--2`
+      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
+    * - start + 4:
+      - G\ :sub:`02low bits 11--4`
+      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`03high bits 13--6`
+      - FG\ :sub:`04low bits 7--0`
+    * - start + 8:
+      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`05high bits 9--2`
+      - G\ :sub:`05high bits 13--10`
+      -
+    * - start + 12:
+      - G\ :sub:`10low bits 7--0`
+      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`11low bits 9--2`
+      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
+    * - start + 16:
+      - FG\ :sub:`12low bits 11--4`
+      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`13high bits 13--6`
+      - R\ :sub:`14low bits 7--0`
+    * - start + 20:
+      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`15high bits 9--2`
+      - FG\ :sub:`15high bits 13--10`
+      -
+    * - start + 24:
+      - B\ :sub:`20low bits 7--0`
+      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`21low bits 9--2`
+      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
+    * - start + 28:
+      - G\ :sub:`22low bits 11--4`
+      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
+      - B\ :sub:`23high bits 13--6`
+      - FG\ :sub:`24low bits 7--0`
+    * - start + 32:
+      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
+      - G\ :sub:`25high bits 9--2`
+      - G\ :sub:`25high bits 13--10`
+      -
+    * - start + 36:
+      - G\ :sub:`30low bits 7--0`
+      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
+      - R\ :sub:`31low bits 9--2`
+      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
+    * - start + 40:
+      - FG\ :sub:`32low bits 11--4`
+      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
+      - G\ :sub:`33high bits 13--6`
+      - R\ :sub:`34low bits 7--0`
+    * - start + 44:
+      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
+      - FG\ :sub:`35high bits 9--2`
+      - FG\ :sub:`35high bits 13--10`
+      -
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
new file mode 100644
index 000000000000..86cadbf38175
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
@@ -0,0 +1,51 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8:
+.. _v4l2-pix-fmt-mtisp-sgbrg8:
+.. _v4l2-pix-fmt-mtisp-sgrbg8:
+.. _v4l2-pix-fmt-mtisp-srggb8:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
+*******************************
+
+8-bit Packed Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format, meaning all the data bits for a pixel lying
+next to each other with no padding in memory, with a depth of 8 bits per pixel.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor.
+They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
+Below is an example of conventional RGB byte order BGGR.
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - G\ :sub:`01`
+      - B\ :sub:`02`
+      - G\ :sub:`03`
+    * - start + 4:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - G\ :sub:`12`
+      - R\ :sub:`13`
+    * - start + 8:
+      - B\ :sub:`20`
+      - G\ :sub:`21`
+      - B\ :sub:`22`
+      - G\ :sub:`23`
+    * - start + 12:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - G\ :sub:`32`
+      - R\ :sub:`33`
\ No newline at end of file
diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
new file mode 100644
index 000000000000..ca5151312bca
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
@@ -0,0 +1,78 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-pix-fmt-mtisp-sbggr8f:
+.. _v4l2-pix-fmt-mtisp-sgbrg8f:
+.. _v4l2-pix-fmt-mtisp-sgrbg8f:
+.. _v4l2-pix-fmt-mtisp-srggb8f:
+
+*******************************
+V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
+*******************************
+
+8-bit Packed Full-G Bayer formats.
+
+Description
+===========
+
+These four pixel formats are used by Mediatek ISP P1.
+This is a packed format with a depth of 8 bits per sample with every 4 pixels.
+Full-G means 1 more pixel for green channel every 2 pixels.
+The least significant byte is stored at lower memory addresses (little-endian).
+The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
+RGB byte order BGGR.
+
+**Bit-packed representation.**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+
+**Byte Order.**
+Each cell is one byte.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - B\ :sub:`00`
+      - FG\ :sub:`01`
+      - G\ :sub:`02`
+      - B\ :sub:`03`
+      - FG\ :sub:`04`
+      - G\ :sub:`05`
+    * - start + 6:
+      - G\ :sub:`10`
+      - R\ :sub:`11`
+      - FG\ :sub:`12`
+      - G\ :sub:`13`
+      - R\ :sub:`14`
+      - FG\ :sub:`15`
+    * - start + 12:
+      - B\ :sub:`20`
+      - FG\ :sub:`21`
+      - G\ :sub:`22`
+      - B\ :sub:`23`
+      - FG\ :sub:`24`
+      - G\ :sub:`25`
+    * - start + 18:
+      - G\ :sub:`30`
+      - R\ :sub:`31`
+      - FG\ :sub:`32`
+      - G\ :sub:`33`
+      - R\ :sub:`34`
+      - FG\ :sub:`35`
\ No newline at end of file
-- 
2.18.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-12-19  5:49   ` Jungo Lin
@ 2019-12-19  5:49     ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, jungo.lin, Pi-Hsun Shih

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
---
Changes from v6:
 - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
 - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
 - Correct auto suspend timer value for suspend/resume issue
 - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
 - Fix KE due to no sen-inf sub-device
---
 drivers/media/platform/mtk-isp/Kconfig        |   20 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 9 files changed, 3377 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..f86e1b59ad1e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,20 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose Y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
+
+		To compile this driver as a module, choose M here; the module
+		will be called mtk-cam-isp.
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..4065d0d29b7f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/remoteproc/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
+#define MTK_ISP_IPI_SEND_TIMEOUT		1000
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	struct device *dev = p1_dev->dev;
+	struct mtk_cam_dev_request *req;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+
+	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
+	if (req)
+		req->timestamp = ktime_get_boottime_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp = scp_get(pdev);
+	if (!p1_dev->scp) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr) {
+		ret = -ENOMEM;
+		goto fail_put_scp;
+	}
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+fail_put_scp:
+	scp_put(p1_dev->scp);
+
+	return ret;
+}
+
+static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
+{
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+	scp_put(p1_dev->scp);
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle if possible */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+				  (val & TG_CS_MASK) == TG_IDLE_ST,
+				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret) {
+		dev_err(dev, "failed to force suspend:%d\n", ret);
+		goto reenable_hw;
+	}
+
+	return 0;
+
+reenable_hw:
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return ret;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret) {
+		isp_teardown_scp_rproc(p1_dev);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	isp_teardown_scp_rproc(p1_dev);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..837662f92a5e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct mtk_scp *scp;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..23fdb8b4abc5
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2087 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 ts_eof = ktime_get_boottime_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = ts_eof;
+		else
+			vb->timestamp = req->timestamp;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			spin_unlock_irqrestore(&cam->running_job_lock, flags);
+			return req;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+
+	return NULL;
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
+	vb2_request_queue(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	mtk_cam_dev_req_try_queue(cam);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	vb2_queue_release(&node->vbq);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "No seninf subdev\n");
+		return -ENODEV;
+	}
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..0a340a1e65ea
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @timestamp: Start of frame timestamp in ns
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	u64 timestamp;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2019-12-19  5:49     ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2019-12-19  5:49 UTC (permalink / raw)
  To: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	jungo.lin, sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

This patch adds the Mediatek ISP P1 HW control device driver.
It handles the ISP HW configuration, provides interrupt handling and
initializes the V4L2 device nodes and other V4L2 functions. Moreover,
implement standard V4L2 video driver that utilizes V4L2 and media
framework APIs. It supports one media device, one sub-device and
several video devices during initialization. Moreover, it also connects
with sensor and seninf drivers with V4L2 async APIs. Communicate with
co-process via SCP communication to compose ISP registers in the
firmware.

(The current metadata interface used in meta input and partial
meta nodes is only a temporary solution to kick off the driver
development and is not ready to be reviewed yet.)

Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
---
Changes from v6:
 - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
 - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
 - Correct auto suspend timer value for suspend/resume issue
 - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
 - Fix KE due to no sen-inf sub-device
---
 drivers/media/platform/mtk-isp/Kconfig        |   20 +
 .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
 .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
 .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
 .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
 .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
 9 files changed, 3377 insertions(+)
 create mode 100644 drivers/media/platform/mtk-isp/Kconfig
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
 create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
new file mode 100644
index 000000000000..f86e1b59ad1e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/Kconfig
@@ -0,0 +1,20 @@
+config VIDEO_MEDIATEK_ISP_PASS1
+	tristate "Mediatek ISP Pass 1 driver"
+	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+	depends on ARCH_MEDIATEK
+	select V4L2_FWNODE
+	select VIDEOBUF2_VMALLOC
+	select VIDEOBUF2_DMA_CONTIG
+	select MTK_SCP
+	default n
+	help
+		Pass 1 driver controls 3A (auto-focus, exposure,
+		and white balance) with tuning feature and outputs
+		the captured image buffers in Mediatek's camera system.
+
+		Choose Y if you want to use Mediatek SoCs to create image
+		captured application such as video recording and still image
+		capturing.
+
+		To compile this driver as a module, choose M here; the module
+		will be called mtk-cam-isp.
diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
new file mode 100644
index 000000000000..ce79d283b209
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
new file mode 100644
index 000000000000..53b54d3c26a0
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+mtk-cam-isp-objs += mtk_cam.o
+mtk-cam-isp-objs += mtk_cam-hw.o
+
+obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
\ No newline at end of file
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
new file mode 100644
index 000000000000..4065d0d29b7f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/atomic.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/remoteproc/mtk_scp.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <linux/vmalloc.h>
+
+#include <media/v4l2-event.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+#include "mtk_cam-regs.h"
+
+#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
+#define MTK_ISP_CQ_BUFFER_COUNT			3
+#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
+
+/*
+ *
+ * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
+ * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
+ * For CAM A/C, it only supports max line buffer with 3328 pixels.
+ * In current driver, only supports CAM B.
+ *
+ */
+#define MTK_ISP_CAM_ID_B			3
+#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
+#define MTK_ISP_IPI_SEND_TIMEOUT		1000
+#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
+
+static void isp_tx_frame_worker(struct work_struct *work)
+{
+	struct mtk_cam_dev_request *req =
+		container_of(work, struct mtk_cam_dev_request, frame_work);
+	struct mtk_cam_dev *cam =
+		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
+		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_handler(void *data, unsigned int len, void *priv)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
+	struct device *dev = p1_dev->dev;
+	struct mtk_isp_scp_p1_cmd *ipi_msg;
+
+	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
+
+	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
+		dev_err(dev, "wrong IPI len:%d\n", len);
+		return;
+	}
+
+	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
+	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
+		return;
+
+	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
+	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
+}
+
+static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct device *dev = p1_dev->dev;
+	int ret;
+
+	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI cmd\n");
+		return ret;
+	}
+	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
+			       isp_composer_handler, p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to register IPI frame\n");
+		goto unreg_ipi_cmd;
+	}
+
+	p1_dev->composer_wq =
+		alloc_ordered_workqueue(dev_name(p1_dev->dev),
+					__WQ_LEGACY | WQ_MEM_RECLAIM |
+					WQ_FREEZABLE);
+	if (!p1_dev->composer_wq) {
+		dev_err(dev, "failed to alloc composer workqueue\n");
+		goto unreg_ipi_frame;
+	}
+
+	return 0;
+
+unreg_ipi_frame:
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
+unreg_ipi_cmd:
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
+
+	return ret;
+}
+
+static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
+{
+	destroy_workqueue(p1_dev->composer_wq);
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
+	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
+}
+
+static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
+	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
+
+	/*
+	 * Passed coherent reserved memory info. for SCP firmware usage.
+	 * This buffer is used for SCP's ISP composer to compose.
+	 * The size of is fixed to 0x200000 for the requirement of composer.
+	 */
+	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
+	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+
+	isp_composer_uninit(p1_dev);
+}
+
+void mtk_isp_hw_config(struct mtk_cam_dev *cam,
+		       struct p1_config_param *config_param)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
+	memcpy(&composer_tx_cmd.config_param, config_param,
+	       sizeof(*config_param));
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
+{
+	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
+	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
+	composer_tx_cmd.is_stream_on = on;
+
+	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
+		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
+}
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	ret = rproc_boot(p1_dev->rproc_handle);
+	if (ret) {
+		dev_err(dev, "failed to rproc_boot\n");
+		return ret;
+	}
+
+	ret = isp_composer_init(p1_dev);
+	if (ret)
+		return ret;
+
+	pm_runtime_get_sync(dev);
+	isp_composer_hw_init(p1_dev);
+
+	p1_dev->enqueued_frame_seq_no = 0;
+	p1_dev->dequeued_frame_seq_no = 0;
+	p1_dev->composed_frame_seq_no = 0;
+	p1_dev->sof_count = 0;
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+int mtk_isp_hw_release(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	isp_composer_hw_deinit(p1_dev);
+	pm_runtime_mark_last_busy(dev);
+	pm_runtime_put_autosuspend(dev);
+	rproc_shutdown(p1_dev->rproc_handle);
+
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
+			 struct mtk_cam_dev_request *req)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
+
+	/* Accumulated frame sequence number */
+	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
+
+	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
+	queue_work(p1_dev->composer_wq, &req->frame_work);
+	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no,
+		cam->running_job_count);
+}
+
+static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
+			       unsigned int dequeued_frame_seq_no)
+{
+	dma_addr_t base_addr = p1_dev->composer_iova;
+	struct device *dev = p1_dev->dev;
+	struct mtk_cam_dev_request *req;
+	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
+	unsigned int addr_offset;
+
+	/* Send V4L2_EVENT_FRAME_SYNC event */
+	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
+
+	p1_dev->sof_count += 1;
+	/* Save frame information */
+	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
+
+	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
+	if (req)
+		req->timestamp = ktime_get_boottime_ns();
+
+	/* Update CQ base address if needed */
+	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
+		dev_dbg(dev,
+			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
+			composed_frame_seq_no, dequeued_frame_seq_no);
+		return;
+	}
+	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
+		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
+	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
+	dev_dbg(dev,
+		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
+		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
+}
+
+static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
+{
+	u32 val;
+
+	dev_err(p1_dev->dev,
+		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
+		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
+		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
+		readl(p1_dev->regs + REG_AAO_ERR_STAT),
+		readl(p1_dev->regs + REG_AFO_ERR_STAT),
+		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
+	dev_err(p1_dev->dev,
+		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
+		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
+		readl(p1_dev->regs + REG_PSO_ERR_STAT),
+		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
+		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
+		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
+
+	/* Disable DMA error mask to avoid too much error log */
+	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
+	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
+	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
+}
+
+static irqreturn_t isp_irq_cam(int irq, void *data)
+{
+	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
+	struct device *dev = p1_dev->dev;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int irq_status, err_status, dma_status;
+	unsigned long flags;
+
+	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
+	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
+	err_status = irq_status & INT_ST_MASK_CAM_ERR;
+	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
+	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
+	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
+
+	/*
+	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
+	 * If these two ISRs come together, print warning msg to hint.
+	 */
+	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
+		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
+
+	/* De-queue frame */
+	if (irq_status & SW_PASS1_DON_ST) {
+		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
+					      p1_dev->dequeued_frame_seq_no);
+		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
+	}
+
+	/* Save frame info. & update CQ address for frame HW en-queue */
+	if (irq_status & SOF_INT_ST)
+		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
+
+	/* Check ISP error status */
+	if (err_status) {
+		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
+		/* Show DMA errors in detail */
+		if (err_status & DMA_ERR_ST)
+			isp_irq_handle_dma_err(p1_dev);
+	}
+
+	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
+		p1_dev->sof_count, irq_status, dma_status,
+		dequeued_frame_seq_no);
+
+	return IRQ_HANDLED;
+}
+
+static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
+			       struct platform_device *pdev)
+{
+	struct device *dev = p1_dev->dev;
+	dma_addr_t addr;
+	void *ptr;
+	int ret;
+
+	p1_dev->scp = scp_get(pdev);
+	if (!p1_dev->scp) {
+		dev_err(dev, "failed to get scp device\n");
+		return -ENODEV;
+	}
+
+	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
+	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
+	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
+
+	/*
+	 * Allocate coherent reserved memory for SCP firmware usage.
+	 * The size of SCP composer's memory is fixed to 0x200000
+	 * for the requirement of firmware.
+	 */
+	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
+				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
+	if (!ptr) {
+		ret = -ENOMEM;
+		goto fail_put_scp;
+	}
+
+	p1_dev->composer_scp_addr = addr;
+	p1_dev->composer_virt_addr = ptr;
+	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
+
+	/*
+	 * This reserved memory is also be used by ISP P1 HW.
+	 * Need to get iova address for ISP P1 DMA.
+	 */
+	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
+				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map scp iova\n");
+		ret = -ENOMEM;
+		goto fail_free_mem;
+	}
+	p1_dev->composer_iova = addr;
+	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
+
+	return 0;
+
+fail_free_mem:
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+fail_put_scp:
+	scp_put(p1_dev->scp);
+
+	return ret;
+}
+
+static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
+{
+	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
+			  p1_dev->composer_virt_addr,
+			  p1_dev->composer_scp_addr);
+	p1_dev->composer_scp_addr = 0;
+	scp_put(p1_dev->scp);
+}
+
+static int mtk_isp_pm_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Disable ISP's view finder and wait for TG idle if possible */
+	dev_dbg(dev, "cam suspend, disable VF\n");
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
+	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
+				  (val & TG_CS_MASK) == TG_IDLE_ST,
+				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
+
+	/* Disable CMOS */
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Force ISP HW to idle */
+	ret = pm_runtime_force_suspend(dev);
+	if (ret) {
+		dev_err(dev, "failed to force suspend:%d\n", ret);
+		goto reenable_hw;
+	}
+
+	return 0;
+
+reenable_hw:
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return ret;
+}
+
+static int mtk_isp_pm_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	u32 val;
+	int ret;
+
+	dev_dbg(dev, "- %s\n", __func__);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	/* Force ISP HW to resume */
+	ret = pm_runtime_force_resume(dev);
+	if (ret)
+		return ret;
+
+	/* Enable CMOS */
+	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
+	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
+	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
+
+	/* Enable VF */
+	val = readl(p1_dev->regs + REG_TG_VF_CON);
+	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_suspend(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	dev_dbg(dev, "%s:disable clock\n", __func__);
+	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
+
+	return 0;
+}
+
+static int mtk_isp_runtime_resume(struct device *dev)
+{
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+	int ret;
+
+	dev_dbg(dev, "%s:enable clock\n", __func__);
+	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to enable clock:%d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_probe(struct platform_device *pdev)
+{
+	/* List of clocks required by isp cam */
+	static const char * const clk_names[] = {
+		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
+	};
+	struct mtk_isp_p1_device *p1_dev;
+	struct device *dev = &pdev->dev;
+	struct resource *res;
+	int irq, ret, i;
+
+	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
+	if (!p1_dev)
+		return -ENOMEM;
+
+	p1_dev->dev = dev;
+	dev_set_drvdata(dev, p1_dev);
+
+	/*
+	 * Now only support single CAM with CAM B.
+	 * Get CAM B register base with CAM B index.
+	 * Support multiple CAMs in future.
+	 */
+	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
+	p1_dev->regs = devm_ioremap_resource(dev, res);
+	if (IS_ERR(p1_dev->regs)) {
+		dev_err(dev, "failed to map reister base\n");
+		return PTR_ERR(p1_dev->regs);
+	}
+	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
+
+	/*
+	 * The cam_sys unit only supports reg., but has no IRQ support.
+	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
+	 */
+	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
+	if (!irq) {
+		dev_err(dev, "failed to get irq\n");
+		return -ENODEV;
+	}
+	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
+			       p1_dev);
+	if (ret) {
+		dev_err(dev, "failed to request irq=%d\n", irq);
+		return ret;
+	}
+	dev_dbg(dev, "registered irq=%d\n", irq);
+	spin_lock_init(&p1_dev->spinlock_irq);
+
+	p1_dev->num_clks = ARRAY_SIZE(clk_names);
+	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
+				    sizeof(*p1_dev->clks), GFP_KERNEL);
+	if (!p1_dev->clks)
+		return -ENOMEM;
+
+	for (i = 0; i < p1_dev->num_clks; ++i)
+		p1_dev->clks[i].id = clk_names[i];
+
+	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
+	if (ret) {
+		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
+		return ret;
+	}
+
+	ret = isp_setup_scp_rproc(p1_dev, pdev);
+	if (ret)
+		return ret;
+
+	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_enable(dev);
+
+	/* Initialize the v4l2 common part */
+	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
+	if (ret) {
+		isp_teardown_scp_rproc(p1_dev);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int mtk_isp_remove(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
+
+	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
+	pm_runtime_dont_use_autosuspend(dev);
+	pm_runtime_disable(dev);
+	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
+			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+	isp_teardown_scp_rproc(p1_dev);
+
+	return 0;
+}
+
+static const struct dev_pm_ops mtk_isp_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
+	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
+			   NULL)
+};
+
+static const struct of_device_id mtk_isp_of_ids[] = {
+	{.compatible = "mediatek,mt8183-camisp",},
+	{}
+};
+MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
+
+static struct platform_driver mtk_isp_driver = {
+	.probe   = mtk_isp_probe,
+	.remove  = mtk_isp_remove,
+	.driver  = {
+		.name  = "mtk-cam-p1",
+		.of_match_table = of_match_ptr(mtk_isp_of_ids),
+		.pm     = &mtk_isp_pm_ops,
+	}
+};
+
+module_platform_driver(mtk_isp_driver);
+
+MODULE_DESCRIPTION("Mediatek ISP P1 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
new file mode 100644
index 000000000000..837662f92a5e
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_HW_H__
+#define __MTK_CAM_HW_H__
+
+#include <linux/types.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ipi.h"
+
+/*
+ * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
+ *
+ * @dev: Pointer to device.
+ * @scp_pdev: Pointer to SCP platform device.
+ * @rproc_handle: Pointer to new remoteproc instance.
+ * @cam_dev: Embedded struct cam_dev
+ * @regs: Camera ISP HW base register address
+ * @num_clks: The number of driver's clocks
+ * @clks: The clock data array
+ * @spinlock_irq: Used to protect register read/write data
+ * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
+ * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
+ * @composed_frame_seq_no: Frame sequence number of composed frame
+ * @timestamp: Frame timestamp in ns
+ * @sof_count: SOF counter
+ * @composer_wq: The work queue for frame request composing
+ * @composer_scp_addr: SCP address of ISP composer memory
+ * @composer_iova: DMA address of ISP composer memory
+ * @virt_addr: Virtual address of ISP composer memory
+ *
+ */
+struct mtk_isp_p1_device {
+	struct device *dev;
+	struct mtk_scp *scp;
+	struct rproc *rproc_handle;
+	struct mtk_cam_dev cam_dev;
+	void __iomem *regs;
+	unsigned int num_clks;
+	struct clk_bulk_data *clks;
+	/* Used to protect register read/write data */
+	spinlock_t spinlock_irq;
+	unsigned int enqueued_frame_seq_no;
+	unsigned int dequeued_frame_seq_no;
+	unsigned int composed_frame_seq_no;
+	u8 sof_count;
+	struct workqueue_struct *composer_wq;
+	dma_addr_t composer_scp_addr;
+	dma_addr_t composer_iova;
+	void *composer_virt_addr;
+};
+
+int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
+int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
+void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
+		       struct p1_config_param *config_param);
+void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
+void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
+			 struct mtk_cam_dev_request *req);
+
+#endif /* __MTK_CAM_HW_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
new file mode 100644
index 000000000000..981b634dd91f
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
@@ -0,0 +1,222 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_IPI_H__
+#define __MTK_CAM_IPI_H__
+
+#include <linux/types.h>
+
+/*
+ * struct img_size - Image size information.
+ *
+ * @w: Image width, the unit is pixel
+ * @h: Image height, the unit is pixel
+ * @xsize: Bytes per line based on width.
+ * @stride: Bytes per line when changing line.
+ *          Stride is based on xsize + HW constrain(byte align).
+ *
+ */
+struct img_size {
+	u32 w;
+	u32 h;
+	u32 xsize;
+	u32 stride;
+} __packed;
+
+/*
+ * struct p1_img_crop - image corp information
+ *
+ * @left: The left of crop area.
+ * @top: The top of crop area.
+ * @width: The width of crop area.
+ * @height: The height of crop area.
+ *
+ */
+struct p1_img_crop {
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+} __packed;
+
+/*
+ * struct dma_buffer - DMA buffer address information
+ *
+ * @iova: DMA address for ISP DMA device
+ * @scp_addr: SCP address for external co-process unit
+ *
+ */
+struct dma_buffer {
+	u32 iova;
+	u32 scp_addr;
+} __packed;
+
+/*
+ * struct p1_img_output - ISP P1 image output information
+ *
+ * @buffer: DMA buffer address of image.
+ * @size: The image size configuration.
+ * @crop: The crop configuration.
+ * @pixel_bits: The bits per image pixel.
+ * @img_fmt: The image format.
+ *
+ */
+struct p1_img_output {
+	struct dma_buffer buffer;
+	struct img_size size;
+	struct p1_img_crop crop;
+	u8 pixel_bits;
+	u32 img_fmt;
+} __packed;
+
+/*
+ * struct cfg_in_param - Image input parameters structure.
+ *                       Normally, it comes from sensor information.
+ *
+ * @continuous: Indicate the sensor mode. Continuous or single shot.
+ * @subsample: Indicate to enables SOF subsample or not.
+ * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
+ * @data_pattern: Describe input data pattern.
+ * @raw_pixel_id: Bayer sequence.
+ * @tg_fps: The fps rate of TG (time generator).
+ * @img_fmt: The image format of input source.
+ * @p1_img_crop: The crop configuration of input source.
+ *
+ */
+struct cfg_in_param {
+	u8 continuous;
+	u8 subsample;
+	u8 pixel_mode;
+	u8 data_pattern;
+	u8 raw_pixel_id;
+	u16 tg_fps;
+	u32 img_fmt;
+	struct p1_img_crop crop;
+} __packed;
+
+/*
+ * struct cfg_main_out_param - The image output parameters of main stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @pure_raw: Indicate the image path control.
+ *            True: pure raw
+ *            False: processing raw
+ * @pure_raw_pack: Indicate the image is packed or not.
+ *                 True: packed mode
+ *                 False: unpacked mode
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_main_out_param {
+	u8 bypass;
+	u8 pure_raw;
+	u8 pure_raw_pack;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct cfg_resize_out_param - The image output parameters of
+ *                               packed out stream.
+ *
+ * @bypass: Indicate this device is enabled or disabled or not.
+ * @p1_img_output: The output image information.
+ *
+ */
+struct cfg_resize_out_param {
+	u8 bypass;
+	struct p1_img_output output;
+} __packed;
+
+/*
+ * struct p1_config_param - ISP P1 configuration parameters.
+ *
+ * @cfg_in_param: The Image input parameters.
+ * @cfg_main_param: The main output image parameters.
+ * @cfg_resize_out_param: The packed output image parameters.
+ * @enabled_dmas: The enabled DMA port information.
+ *
+ */
+struct p1_config_param {
+	struct cfg_in_param cfg_in_param;
+	struct cfg_main_out_param cfg_main_param;
+	struct cfg_resize_out_param cfg_resize_param;
+	u32 enabled_dmas;
+} __packed;
+
+/*
+ * struct P1_meta_frame - ISP P1 meta frame information.
+ *
+ * @enabled_dma: The enabled DMA port information.
+ * @vb_index: The VB2 index of meta buffer.
+ * @meta_addr: DMA buffer address of meta buffer.
+ *
+ */
+struct P1_meta_frame {
+	u32 enabled_dma;
+	u32 vb_index;
+	struct dma_buffer meta_addr;
+} __packed;
+
+/*
+ * struct isp_init_info - ISP P1 composer init information.
+ *
+ * @hw_module: The ISP Camera HW module ID.
+ * @cq_addr: The DMA address of composer memory.
+ *
+ */
+struct isp_init_info {
+	u8 hw_module;
+	struct dma_buffer cq_addr;
+} __packed;
+
+/*
+ * struct isp_ack_info - ISP P1 IPI command ack information.
+ *
+ * @cmd_id: The IPI command ID is acked.
+ * @frame_seq_no: The IPI frame sequence number is acked.
+ *
+ */
+struct isp_ack_info {
+	u8 cmd_id;
+	u32 frame_seq_no;
+} __packed;
+
+/*
+ * The IPI command enumeration.
+ */
+enum mtk_isp_scp_cmds {
+	ISP_CMD_INIT,
+	ISP_CMD_CONFIG,
+	ISP_CMD_STREAM,
+	ISP_CMD_DEINIT,
+	ISP_CMD_ACK,
+	ISP_CMD_FRAME_ACK,
+	ISP_CMD_RESERVED,
+};
+
+/*
+ * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
+ *
+ * @cmd_id: The IPI command ID.
+ * @init_param: The init formation for ISP_CMD_INIT.
+ * @config_param: The cmd configuration for ISP_CMD_CONFIG.
+ * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
+ * @is_stream_on: The stream information for ISP_CMD_STREAM.
+ * @ack_info: The cmd ack. information for ISP_CMD_ACK.
+ *
+ */
+struct mtk_isp_scp_p1_cmd {
+	u8 cmd_id;
+	union {
+		struct isp_init_info init_param;
+		struct p1_config_param config_param;
+		u32 enabled_dmas;
+		struct P1_meta_frame meta_frame;
+		u8 is_stream_on;
+		struct isp_ack_info ack_info;
+	};
+} __packed;
+
+#endif /* __MTK_CAM_IPI_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
new file mode 100644
index 000000000000..ab2277f45fa4
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_REGS_H__
+#define __MTK_CAM_REGS_H__
+
+/* ISP interrupt enable */
+#define REG_CTL_RAW_INT_EN		0x0020
+#define DMA_ERR_INT_EN			BIT(29)
+
+/* ISP interrupt status */
+#define REG_CTL_RAW_INT_STAT		0x0024
+#define VS_INT_ST			BIT(0)
+#define TG_ERR_ST			BIT(4)
+#define TG_GBERR_ST			BIT(5)
+#define CQ_CODE_ERR_ST			BIT(6)
+#define CQ_APB_ERR_ST			BIT(7)
+#define CQ_VS_ERR_ST			BIT(8)
+#define HW_PASS1_DON_ST			BIT(11)
+#define SOF_INT_ST			BIT(12)
+#define AMX_ERR_ST			BIT(15)
+#define RMX_ERR_ST			BIT(16)
+#define BMX_ERR_ST			BIT(17)
+#define RRZO_ERR_ST			BIT(18)
+#define AFO_ERR_ST			BIT(19)
+#define IMGO_ERR_ST			BIT(20)
+#define AAO_ERR_ST			BIT(21)
+#define PSO_ERR_ST			BIT(22)
+#define LCSO_ERR_ST			BIT(23)
+#define BNR_ERR_ST			BIT(24)
+#define LSCI_ERR_ST			BIT(25)
+#define DMA_ERR_ST			BIT(29)
+#define SW_PASS1_DON_ST			BIT(30)
+
+/* ISP interrupt 2 status */
+#define REG_CTL_RAW_INT2_STAT		0x0034
+#define AFO_DONE_ST			BIT(5)
+#define AAO_DONE_ST			BIT(7)
+
+/* Configures sensor mode */
+#define REG_TG_SEN_MODE			0x0230
+#define TG_SEN_MODE_CMOS_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_VF_CON			0x0234
+#define TG_VF_CON_VFDATA_EN		BIT(0)
+
+/* View finder mode control */
+#define REG_TG_INTER_ST			0x026c
+#define TG_CS_MASK			0x3f00
+#define TG_IDLE_ST			BIT(8)
+
+/* IMGO error status register */
+#define REG_IMGO_ERR_STAT		0x1360
+/* RRZO error status register */
+#define REG_RRZO_ERR_STAT		0x1364
+/* AAO error status register */
+#define REG_AAO_ERR_STAT		0x1368
+/* AFO error status register */
+#define REG_AFO_ERR_STAT		0x136c
+/* LCSO error status register */
+#define REG_LCSO_ERR_STAT		0x1370
+/* BPCI error status register */
+#define REG_BPCI_ERR_STAT		0x137c
+/* LSCI error status register */
+#define REG_LSCI_ERR_STAT		0x1384
+/* LMVO error status register */
+#define REG_LMVO_ERR_STAT		0x1390
+/* FLKO error status register */
+#define REG_FLKO_ERR_STAT		0x1394
+/* PSO error status register */
+#define REG_PSO_ERR_STAT		0x13a0
+
+/* CQ0 base address */
+#define REG_CQ_THR0_BASEADDR		0x0198
+/* Frame sequence number */
+#define REG_FRAME_SEQ_NUM		0x13b8
+
+/* IRQ Error Mask */
+#define INT_ST_MASK_CAM_ERR		( \
+					TG_ERR_ST |\
+					TG_GBERR_ST |\
+					CQ_CODE_ERR_ST |\
+					CQ_APB_ERR_ST |\
+					CQ_VS_ERR_ST |\
+					BNR_ERR_ST |\
+					RMX_ERR_ST |\
+					BMX_ERR_ST |\
+					BNR_ERR_ST |\
+					LSCI_ERR_ST |\
+					DMA_ERR_ST)
+
+#endif	/* __MTK_CAM_REGS_H__ */
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
new file mode 100644
index 000000000000..23fdb8b4abc5
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
@@ -0,0 +1,2087 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 MediaTek Inc.
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-hw.h"
+
+#define R_IMGO		BIT(0)
+#define R_RRZO		BIT(1)
+#define R_AAO		BIT(3)
+#define R_AFO		BIT(4)
+#define R_LCSO		BIT(5)
+#define R_LMVO		BIT(7)
+#define R_FLKO		BIT(8)
+#define R_PSO		BIT(10)
+
+#define MTK_ISP_ONE_PIXEL_MODE		1
+#define MTK_ISP_MIN_RESIZE_RATIO	6
+#define MTK_ISP_MAX_RUNNING_JOBS	3
+
+#define MTK_CAM_CIO_PAD_SRC		4
+#define MTK_CAM_CIO_PAD_SINK		11
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+	return container_of(video_devdata(__file),
+		struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+	return container_of(__vq, struct mtk_cam_video_device, vbq);
+}
+
+static inline struct mtk_cam_dev_request *
+mtk_cam_req_to_dev_req(struct media_request *__req)
+{
+	return container_of(__req, struct mtk_cam_dev_request, req);
+}
+
+static inline struct mtk_cam_dev_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
+}
+
+static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
+				 struct mtk_cam_dev_request *req,
+				 enum vb2_buffer_state state)
+{
+	struct media_request_object *obj, *obj_prev;
+	unsigned long flags;
+	u64 ts_eof = ktime_get_boottime_ns();
+
+	if (!cam->streaming)
+		return;
+
+	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
+		req->req.debug_str, req->frame_params.frame_seq_no, state);
+
+	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
+		struct vb2_buffer *vb;
+		struct mtk_cam_dev_buffer *buf;
+		struct mtk_cam_video_device *node;
+
+		if (!vb2_request_object_is_buffer(obj))
+			continue;
+		vb = container_of(obj, struct vb2_buffer, req_obj);
+		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+		spin_lock_irqsave(&node->buf_list_lock, flags);
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->buf_list_lock, flags);
+		buf->vbb.sequence = req->frame_params.frame_seq_no;
+		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+			vb->timestamp = ts_eof;
+		else
+			vb->timestamp = req->timestamp;
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+}
+
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			spin_unlock_irqrestore(&cam->running_job_lock, flags);
+			return req;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+
+	return NULL;
+}
+
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
+				   unsigned int frame_seq_no)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
+		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
+			req->frame_params.frame_seq_no, frame_seq_no);
+
+		/* Match by the en-queued request number */
+		if (req->frame_params.frame_seq_no == frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
+			list_del(&req->list);
+			break;
+		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
+			cam->running_job_count--;
+			/* Pass to user space for frame drop */
+			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
+			dev_warn(cam->dev, "frame_seq:%d drop\n",
+				 req->frame_params.frame_seq_no);
+			list_del(&req->list);
+		} else {
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
+		list_del(&req->list);
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+}
+
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
+{
+	struct mtk_cam_dev_request *req, *req_prev;
+	unsigned long flags;
+
+	if (!cam->streaming) {
+		dev_dbg(cam->dev, "stream is off\n");
+		return;
+	}
+
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	spin_lock_irqsave(&cam->running_job_lock, flags);
+	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
+		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
+			dev_dbg(cam->dev, "jobs are full\n");
+			break;
+		}
+		cam->running_job_count++;
+		list_del(&req->list);
+		list_add_tail(&req->list, &cam->running_job_list);
+		mtk_isp_req_enqueue(cam, req);
+	}
+	spin_unlock_irqrestore(&cam->running_job_lock, flags);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+}
+
+static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
+{
+	struct mtk_cam_dev_request *cam_dev_req;
+
+	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
+
+	return &cam_dev_req->req;
+}
+
+static void mtk_cam_req_free(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
+
+	kfree(cam_dev_req);
+}
+
+static void mtk_cam_req_queue(struct media_request *req)
+{
+	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
+	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
+					       media_dev);
+	unsigned long flags;
+
+	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
+	vb2_request_queue(req);
+
+	/* add to pending job list */
+	spin_lock_irqsave(&cam->pending_job_lock, flags);
+	list_add_tail(&cam_req->list, &cam->pending_job_list);
+	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
+
+	mtk_cam_dev_req_try_queue(cam);
+}
+
+static unsigned int get_pixel_bits(unsigned int pix_fmt)
+{
+	switch (pix_fmt) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return 8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return 10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return 12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return 14;
+	default:
+		return 0;
+	}
+}
+
+static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
+			     struct v4l2_pix_format_mplane *mp)
+{
+	unsigned int bpl, ppl;
+	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
+	unsigned int width = mp->width;
+
+	bpl = 0;
+	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
+		/* Bayer encoding format & 2 bytes alignment */
+		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
+	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
+		/*
+		 * The FULL-G encoding format
+		 * 1 G component per pixel
+		 * 1 R component per 4 pixel
+		 * 1 B component per 4 pixel
+		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
+		 */
+		ppl = DIV_ROUND_UP(width * 6, 4);
+		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
+
+		/* 4 bytes alignment for 10 bit & others are 8 bytes */
+		if (pixel_bits == 10)
+			bpl = ALIGN(bpl, 4);
+		else
+			bpl = ALIGN(bpl, 8);
+	}
+	/*
+	 * This image output buffer will be input buffer of MTK CAM DIP HW
+	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
+	 */
+	bpl = ALIGN(bpl, 4);
+
+	mp->plane_fmt[0].bytesperline = bpl;
+	mp->plane_fmt[0].sizeimage = bpl * mp->height;
+
+	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
+		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
+}
+
+static const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+	int i;
+	const struct v4l2_format *dev_fmt;
+
+	for (i = 0; i < desc->num_fmts; i++) {
+		dev_fmt = &desc->fmts[i];
+		if (dev_fmt->fmt.pix_mp.pixelformat == format)
+			return dev_fmt;
+	}
+
+	return NULL;
+}
+
+/* Get the default format setting */
+static void
+mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
+			     struct mtk_cam_dev_node_desc *queue_desc,
+			     struct v4l2_format *dest)
+{
+	const struct v4l2_format *default_fmt =
+		&queue_desc->fmts[queue_desc->default_fmt_idx];
+
+	dest->type = queue_desc->buf_type;
+
+	/* Configure default format based on node type */
+	if (!queue_desc->image) {
+		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+		return;
+	}
+
+	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
+	dest->fmt.pix_mp.num_planes = 1;
+
+	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+}
+
+/* Utility functions */
+static unsigned int get_sensor_pixel_id(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+		return MTK_CAM_RAW_PXL_ID_B;
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GB;
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+		return MTK_CAM_RAW_PXL_ID_GR;
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_RAW_PXL_ID_R;
+	default:
+		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
+	}
+}
+
+static unsigned int get_sensor_fmt(unsigned int fmt)
+{
+	switch (fmt) {
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case MEDIA_BUS_FMT_SBGGR12_1X12:
+	case MEDIA_BUS_FMT_SGBRG12_1X12:
+	case MEDIA_BUS_FMT_SGRBG12_1X12:
+	case MEDIA_BUS_FMT_SRGGB12_1X12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case MEDIA_BUS_FMT_SBGGR14_1X14:
+	case MEDIA_BUS_FMT_SGBRG14_1X14:
+	case MEDIA_BUS_FMT_SGRBG14_1X14:
+	case MEDIA_BUS_FMT_SRGGB14_1X14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static unsigned int get_img_fmt(unsigned int fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_MTISP_SBGGR8:
+	case V4L2_PIX_FMT_MTISP_SGBRG8:
+	case V4L2_PIX_FMT_MTISP_SGRBG8:
+	case V4L2_PIX_FMT_MTISP_SRGGB8:
+		return MTK_CAM_IMG_FMT_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR8F:
+	case V4L2_PIX_FMT_MTISP_SGBRG8F:
+	case V4L2_PIX_FMT_MTISP_SGRBG8F:
+	case V4L2_PIX_FMT_MTISP_SRGGB8F:
+		return MTK_CAM_IMG_FMT_FG_BAYER8;
+	case V4L2_PIX_FMT_MTISP_SBGGR10:
+	case V4L2_PIX_FMT_MTISP_SGBRG10:
+	case V4L2_PIX_FMT_MTISP_SGRBG10:
+	case V4L2_PIX_FMT_MTISP_SRGGB10:
+		return MTK_CAM_IMG_FMT_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR10F:
+	case V4L2_PIX_FMT_MTISP_SGBRG10F:
+	case V4L2_PIX_FMT_MTISP_SGRBG10F:
+	case V4L2_PIX_FMT_MTISP_SRGGB10F:
+		return MTK_CAM_IMG_FMT_FG_BAYER10;
+	case V4L2_PIX_FMT_MTISP_SBGGR12:
+	case V4L2_PIX_FMT_MTISP_SGBRG12:
+	case V4L2_PIX_FMT_MTISP_SGRBG12:
+	case V4L2_PIX_FMT_MTISP_SRGGB12:
+		return MTK_CAM_IMG_FMT_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR12F:
+	case V4L2_PIX_FMT_MTISP_SGBRG12F:
+	case V4L2_PIX_FMT_MTISP_SGRBG12F:
+	case V4L2_PIX_FMT_MTISP_SRGGB12F:
+		return MTK_CAM_IMG_FMT_FG_BAYER12;
+	case V4L2_PIX_FMT_MTISP_SBGGR14:
+	case V4L2_PIX_FMT_MTISP_SGBRG14:
+	case V4L2_PIX_FMT_MTISP_SGRBG14:
+	case V4L2_PIX_FMT_MTISP_SRGGB14:
+		return MTK_CAM_IMG_FMT_BAYER14;
+	case V4L2_PIX_FMT_MTISP_SBGGR14F:
+	case V4L2_PIX_FMT_MTISP_SGBRG14F:
+	case V4L2_PIX_FMT_MTISP_SGRBG14F:
+	case V4L2_PIX_FMT_MTISP_SRGGB14F:
+		return MTK_CAM_IMG_FMT_FG_BAYER14;
+	default:
+		return MTK_CAM_IMG_FMT_UNKNOWN;
+	}
+}
+
+static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
+			  struct p1_img_output *out_fmt, int sd_width,
+			  int sd_height)
+{
+	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
+
+	/* Check output & input image size dimension */
+	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
+	    cfg_fmt->fmt.pix_mp.height > sd_height) {
+		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
+			node_id);
+		return -EINVAL;
+	}
+
+	/* Check resize ratio for resize out stream due to HW constraint */
+	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
+	    MTK_ISP_MIN_RESIZE_RATIO) ||
+	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
+	    MTK_ISP_MIN_RESIZE_RATIO)) {
+		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
+			node_id, MTK_ISP_MIN_RESIZE_RATIO);
+		return -EINVAL;
+	}
+
+	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
+	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
+	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    !out_fmt->pixel_bits) {
+		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
+			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
+		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
+
+	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
+	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
+	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
+
+	out_fmt->crop.left = 0;
+	out_fmt->crop.top = 0;
+	out_fmt->crop.width = sd_width;
+	out_fmt->crop.height = sd_height;
+
+	dev_dbg(cam->dev,
+		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
+		node_id, out_fmt->size.w, out_fmt->size.h,
+		out_fmt->size.stride, out_fmt->size.xsize,
+		out_fmt->crop.width, out_fmt->crop.height);
+
+	return 0;
+}
+
+static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	cam->enabled_count = 0;
+	cam->enabled_dmas = 0;
+	cam->stream_count = 0;
+	cam->running_job_count = 0;
+
+	/* Get the enabled meta DMA ports */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		if (!cam->vdev_nodes[i].enabled)
+			continue;
+		cam->enabled_count++;
+		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
+	}
+
+	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
+		cam->enabled_dmas);
+}
+
+static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	struct p1_config_param config_param;
+	struct cfg_in_param *cfg_in_param;
+	struct v4l2_subdev_format sd_fmt;
+	int sd_width, sd_height, sd_code;
+	unsigned int enabled_dma_ports = cam->enabled_dmas;
+	int ret;
+
+	/* Get sensor format configuration */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
+	if (ret) {
+		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
+		return ret;
+	}
+	sd_width = sd_fmt.format.width;
+	sd_height = sd_fmt.format.height;
+	sd_code = sd_fmt.format.code;
+	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
+		sd_code);
+
+	memset(&config_param, 0, sizeof(config_param));
+
+	/* Update cfg_in_param */
+	cfg_in_param = &config_param.cfg_in_param;
+	cfg_in_param->continuous = true;
+	/* Fix to one pixel mode in default */
+	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
+	cfg_in_param->crop.width = sd_width;
+	cfg_in_param->crop.height = sd_height;
+	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
+	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
+	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
+	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
+		dev_err(dev, "unknown sd code:%d\n", sd_code);
+		return -EINVAL;
+	}
+
+	/* Update cfg_main_param */
+	config_param.cfg_main_param.pure_raw = true;
+	config_param.cfg_main_param.pure_raw_pack = true;
+	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
+			     &config_param.cfg_main_param.output,
+			     sd_width, sd_height);
+	if (ret)
+		return ret;
+
+	/* Update cfg_resize_param */
+	if (enabled_dma_ports & R_RRZO) {
+		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
+				     &config_param.cfg_resize_param.output,
+				     sd_width, sd_height);
+		if (ret)
+			return ret;
+	} else {
+		config_param.cfg_resize_param.bypass = true;
+	}
+
+	/* Update enabled_dmas */
+	config_param.enabled_dmas = enabled_dma_ports;
+	mtk_isp_hw_config(cam, &config_param);
+	dev_dbg(dev, "%s done\n", __func__);
+
+	return 0;
+}
+
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
+				  unsigned int frame_seq_no)
+{
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = frame_seq_no,
+	};
+
+	v4l2_event_queue(cam->subdev.devnode, &event);
+}
+
+static struct v4l2_subdev *
+mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
+{
+	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
+	struct device *dev = cam->dev;
+	struct media_entity *entity;
+	struct v4l2_subdev *sensor;
+
+	sensor = NULL;
+	media_device_for_each_entity(entity, mdev) {
+		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
+			entity->name, entity->function, entity->stream_count);
+		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
+		    entity->stream_count) {
+			sensor = media_entity_to_v4l2_subdev(entity);
+			dev_dbg(dev, "sensor found: %s\n", entity->name);
+			break;
+		}
+	}
+
+	if (!sensor)
+		dev_err(dev, "no seninf connected\n");
+
+	return sensor;
+}
+
+static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "no seninf connected\n");
+		return -ENODEV;
+	}
+
+	/* Get active sensor from graph topology */
+	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
+	if (!cam->sensor)
+		return -ENODEV;
+
+	/* Seninf must stream on first */
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
+	if (ret) {
+		dev_err(dev, "failed to stream on %s:%d\n",
+			cam->sensor->entity.name, ret);
+		goto fail_seninf_off;
+	}
+
+	ret = mtk_cam_dev_isp_config(cam);
+	if (ret)
+		goto fail_sensor_off;
+
+	cam->streaming = true;
+	mtk_isp_stream(cam, 1);
+	mtk_cam_dev_req_try_queue(cam);
+	dev_dbg(dev, "streamed on Pass 1\n");
+
+	return 0;
+
+fail_sensor_off:
+	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+fail_seninf_off:
+	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+
+	return ret;
+}
+
+static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->sensor->entity.name, ret);
+		return -EPERM;
+	}
+
+	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
+	if (ret) {
+		dev_err(dev, "failed to stream off %s:%d\n",
+			cam->seninf->entity.name, ret);
+		return -EPERM;
+	}
+
+	cam->streaming = false;
+	mtk_isp_stream(cam, 0);
+	mtk_isp_hw_release(cam);
+
+	dev_dbg(dev, "streamed off Pass 1\n");
+
+	return 0;
+}
+
+static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
+
+	if (enable) {
+		/* Align vb2_core_streamon design */
+		if (cam->streaming) {
+			dev_warn(cam->dev, "already streaming on\n");
+			return 0;
+		}
+		return mtk_cam_cio_stream_on(cam);
+	}
+
+	if (!cam->streaming) {
+		dev_warn(cam->dev, "already streaming off\n");
+		return 0;
+	}
+	return mtk_cam_cio_stream_off(cam);
+}
+
+static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
+				      struct v4l2_fh *fh,
+				      struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static int mtk_cam_media_link_setup(struct media_entity *entity,
+				    const struct media_pad *local,
+				    const struct media_pad *remote, u32 flags)
+{
+	struct mtk_cam_dev *cam =
+		container_of(entity, struct mtk_cam_dev, subdev.entity);
+	u32 pad = local->index;
+
+	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
+		__func__, pad, remote->index, flags);
+
+	/*
+	 * The video nodes exposed by the driver have pads indexes
+	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
+	 */
+	if (pad < MTK_CAM_P1_TOTAL_NODES)
+		cam->vdev_nodes[pad].enabled =
+			!!(flags & MEDIA_LNK_FL_ENABLED);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	unsigned long flags;
+
+	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
+		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
+
+	/* added the buffer into the tracking list */
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_add_tail(&buf->list, &node->buf_list);
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+
+	/* update buffer internal address */
+	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
+	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct device *dev = cam->dev;
+	struct mtk_cam_dev_buffer *buf;
+	dma_addr_t addr;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	buf->node_id = node->id;
+	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+	buf->scp_addr = 0;
+
+	/* SCP address is only valid for meta input buffer */
+	if (!node->desc.smem_alloc)
+		return 0;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	/* Use coherent address to get iova address */
+	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, addr)) {
+		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
+		return -EFAULT;
+	}
+	buf->scp_addr = buf->daddr;
+	buf->daddr = addr;
+
+	return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+		if (vb2_get_plane_payload(vb, 0) != size) {
+			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
+				vb2_get_plane_payload(vb, 0), size);
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	v4l2_buf->field = V4L2_FIELD_NONE;
+	vb2_set_plane_payload(vb, 0, size);
+
+	return 0;
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+	struct mtk_cam_dev_buffer *buf;
+	struct device *dev = cam->dev;
+
+	if (!node->desc.smem_alloc)
+		return;
+
+	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+	dma_unmap_page_attrs(dev, buf->daddr,
+			     vb->planes[0].length,
+			     DMA_BIDIRECTIONAL,
+			     DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+	dev_dbg(cam->dev, "%s\n", __func__);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+				   unsigned int *num_buffers,
+				   unsigned int *num_planes,
+				   unsigned int sizes[],
+				   struct device *alloc_devs[])
+{
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	unsigned int max_buffer_count = node->desc.max_buf_count;
+	const struct v4l2_format *fmt = &node->vdev_fmt;
+	unsigned int size;
+
+	/* Check the limitation of buffer size */
+	if (max_buffer_count)
+		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+	if (node->desc.smem_alloc)
+		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+		size = fmt->fmt.meta.buffersize;
+	else
+		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+	if (*num_planes) {
+		if (sizes[0] < size || *num_planes != 1)
+			return -EINVAL;
+	} else {
+		*num_planes = 1;
+		sizes[0] = size;
+	}
+
+	return 0;
+}
+
+static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
+					   struct mtk_cam_video_device *node,
+					   enum vb2_buffer_state state)
+{
+	struct mtk_cam_dev_buffer *buf, *buf_prev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&node->buf_list_lock, flags);
+	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vbb.vb2_buf, state);
+	}
+	spin_unlock_irqrestore(&node->buf_list_lock, flags);
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+				       unsigned int count)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!node->enabled) {
+		dev_err(dev, "Node:%d is not enabled\n", node->id);
+		ret = -ENOLINK;
+		goto fail_ret_buf;
+	}
+
+	mutex_lock(&cam->op_lock);
+	/* Start streaming of the whole pipeline now*/
+	if (!cam->pipeline.streaming_count) {
+		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
+		if (ret) {
+			dev_err(dev, "failed to start pipeline:%d\n", ret);
+			goto fail_unlock;
+		}
+		mtk_cam_dev_init_stream(cam);
+		ret = mtk_isp_hw_init(cam);
+		if (ret) {
+			dev_err(dev, "failed to init HW:%d\n", ret);
+			goto fail_stop_pipeline;
+		}
+	}
+
+	/* Media links are fixed after media_pipeline_start */
+	cam->stream_count++;
+	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
+		cam->enabled_count);
+	if (cam->stream_count < cam->enabled_count) {
+		mutex_unlock(&cam->op_lock);
+		return 0;
+	}
+
+	/* Stream on sub-devices node */
+	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
+	if (ret)
+		goto fail_no_stream;
+	mutex_unlock(&cam->op_lock);
+
+	return 0;
+
+fail_no_stream:
+	cam->stream_count--;
+fail_stop_pipeline:
+	if (cam->stream_count == 0)
+		media_pipeline_stop(&node->vdev.entity);
+fail_unlock:
+	mutex_unlock(&cam->op_lock);
+fail_ret_buf:
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
+
+	return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
+	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+	struct device *dev = cam->dev;
+
+	mutex_lock(&cam->op_lock);
+	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
+		cam->stream_count);
+	/* Check the first node to stream-off */
+	if (cam->stream_count == cam->enabled_count)
+		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
+
+	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
+	cam->stream_count--;
+	if (cam->stream_count) {
+		mutex_unlock(&cam->op_lock);
+		return;
+	}
+	mutex_unlock(&cam->op_lock);
+
+	mtk_cam_dev_req_cleanup(cam);
+	media_pipeline_stop(&node->vdev.entity);
+}
+
+static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+
+	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 dev_name(cam->dev));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+				   struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index >= node->desc.num_fmts)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt = node->vdev_fmt.fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+				  struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+	struct device *dev = cam->dev;
+	const struct v4l2_format *dev_fmt;
+	struct v4l2_format try_fmt;
+
+	memset(&try_fmt, 0, sizeof(try_fmt));
+	try_fmt.type = f->type;
+
+	/* Validate pixelformat */
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+	if (!dev_fmt) {
+		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
+		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
+	}
+	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+	/* Validate image width & height range */
+	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+	/* 4 bytes alignment for width */
+	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
+
+	/* Only support one plane */
+	try_fmt.fmt.pix_mp.num_planes = 1;
+
+	/* bytesperline & sizeimage calculation */
+	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
+
+	/* Constant format fields */
+	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+	*f = try_fmt;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+				struct v4l2_format *f)
+{
+	struct mtk_cam_dev *cam = video_drvdata(file);
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (vb2_is_busy(node->vdev.queue)) {
+		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/* Get the valid format */
+	mtk_cam_vidioc_try_fmt(file, fh, f);
+	/* Configure to video device */
+	node->vdev_fmt = *f;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+					  struct v4l2_frmsizeenum *sizes)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+	const struct v4l2_format *dev_fmt;
+
+	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+	if (!dev_fmt || sizes->index)
+		return -EINVAL;
+
+	sizes->type = node->desc.frmsizes->type;
+	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+	       sizeof(sizes->stepwise));
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+					struct v4l2_fmtdesc *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	if (f->index)
+		return -EINVAL;
+
+	/* f->description is filled in v4l_fill_fmtdesc function */
+	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->flags = 0;
+
+	return 0;
+}
+
+static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+				     struct v4l2_format *f)
+{
+	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
+	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
+	.subscribe_event = mtk_cam_sd_subscribe_event,
+	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
+	.s_stream =  mtk_cam_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
+	.core = &mtk_cam_subdev_core_ops,
+	.video = &mtk_cam_subdev_video_ops,
+};
+
+static const struct media_entity_operations mtk_cam_media_entity_ops = {
+	.link_setup = mtk_cam_media_link_setup,
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+	.queue_setup = mtk_cam_vb2_queue_setup,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.buf_init = mtk_cam_vb2_buf_init,
+	.buf_prepare = mtk_cam_vb2_buf_prepare,
+	.start_streaming = mtk_cam_vb2_start_streaming,
+	.stop_streaming = mtk_cam_vb2_stop_streaming,
+	.buf_queue = mtk_cam_vb2_buf_queue,
+	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
+	.buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static const struct media_device_ops mtk_cam_media_ops = {
+	.req_alloc = mtk_cam_req_alloc,
+	.req_free = mtk_cam_req_free,
+	.req_validate = vb2_request_validate,
+	.req_queue = mtk_cam_req_queue,
+};
+
+static int mtk_cam_media_register(struct mtk_cam_dev *cam,
+				  struct media_device *media_dev)
+{
+	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
+	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	media_dev->dev = cam->dev;
+	strscpy(media_dev->model, dev_driver_string(dev),
+		sizeof(media_dev->model));
+	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+		 "platform:%s", dev_name(dev));
+	media_dev->hw_revision = 0;
+	media_device_init(media_dev);
+	media_dev->ops = &mtk_cam_media_ops;
+
+	ret = media_device_register(media_dev);
+	if (ret) {
+		dev_err(dev, "failed to register media device:%d\n", ret);
+		return ret;
+	}
+
+	/* Initialize subdev pads */
+	cam->subdev_pads = devm_kcalloc(dev, num_pads,
+					sizeof(*cam->subdev_pads),
+					GFP_KERNEL);
+	if (!cam->subdev_pads) {
+		dev_err(dev, "failed to allocate subdev_pads\n");
+		ret = -ENOMEM;
+		goto fail_media_unreg;
+	}
+
+	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
+				     cam->subdev_pads);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pads:%d\n", ret);
+		goto fail_media_unreg;
+	}
+
+	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
+	for (i = 0; i < num_pads; i++)
+		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	/* Customize the last one pad as CIO sink pad. */
+	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	return 0;
+
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int
+mtk_cam_video_register_device(struct mtk_cam_dev *cam,
+			      struct mtk_cam_video_device *node)
+{
+	struct device *dev = cam->dev;
+	struct video_device *vdev = &node->vdev;
+	struct vb2_queue *vbq = &node->vbq;
+	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
+	unsigned int link_flags = node->desc.link_flags;
+	int ret;
+
+	/* Initialize mtk_cam_video_device */
+	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
+		node->enabled = true;
+	else
+		node->enabled = false;
+	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
+
+	cam->subdev_pads[node->id].flags = output ?
+		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+	/* Initialize media entities */
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+	if (ret) {
+		dev_err(dev, "failed to initialize media pad:%d\n", ret);
+		return ret;
+	}
+	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+	/* Initialize vbq */
+	vbq->type = node->desc.buf_type;
+	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
+		vbq->io_modes = VB2_MMAP;
+	else
+		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+
+	if (node->desc.smem_alloc) {
+		vbq->bidirectional = 1;
+		vbq->dev = cam->smem_dev;
+	} else {
+		vbq->dev = dev;
+	}
+	vbq->ops = &mtk_cam_vb2_ops;
+	vbq->mem_ops = &vb2_dma_contig_memops;
+	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
+	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
+	if (output)
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+	else
+		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+	/* No minimum buffers limitation */
+	vbq->min_buffers_needed = 0;
+	vbq->drv_priv = cam;
+	vbq->lock = &node->vdev_lock;
+	vbq->supports_requests = true;
+	vbq->requires_requests = true;
+
+	ret = vb2_queue_init(vbq);
+	if (ret) {
+		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
+		goto fail_media_clean;
+	}
+
+	/* Initialize vdev */
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
+		 dev_driver_string(dev), node->desc.name);
+	/* set cap/type/ioctl_ops of the video device */
+	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
+	vdev->ioctl_ops = node->desc.ioctl_ops;
+	vdev->fops = &mtk_cam_v4l2_fops;
+	vdev->release = video_device_release_empty;
+	vdev->lock = &node->vdev_lock;
+	vdev->v4l2_dev = &cam->v4l2_dev;
+	vdev->queue = &node->vbq;
+	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+	vdev->entity.ops = NULL;
+	video_set_drvdata(vdev, cam);
+	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
+
+	/* Initialize miscellaneous variables */
+	mutex_init(&node->vdev_lock);
+	INIT_LIST_HEAD(&node->buf_list);
+	spin_lock_init(&node->buf_list_lock);
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret) {
+		dev_err(dev, "failed to register vde:%d\n", ret);
+		goto fail_vb2_rel;
+	}
+
+	/* Create link between video node and the subdev pad */
+	if (output) {
+		ret = media_create_pad_link(&vdev->entity, 0,
+					    &cam->subdev.entity,
+					    node->id, link_flags);
+	} else {
+		ret = media_create_pad_link(&cam->subdev.entity,
+					    node->id, &vdev->entity, 0,
+					    link_flags);
+	}
+	if (ret)
+		goto fail_vdev_ureg;
+
+	return 0;
+
+fail_vdev_ureg:
+	video_unregister_device(vdev);
+fail_vb2_rel:
+	mutex_destroy(&node->vdev_lock);
+	vb2_queue_release(vbq);
+fail_media_clean:
+	media_entity_cleanup(&vdev->entity);
+
+	return ret;
+}
+
+static void
+mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
+{
+	video_unregister_device(&node->vdev);
+	vb2_queue_release(&node->vbq);
+	media_entity_cleanup(&node->vdev.entity);
+	mutex_destroy(&node->vdev_lock);
+}
+
+static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int i, ret;
+
+	/* Set up media device & pads */
+	ret = mtk_cam_media_register(cam, &cam->media_dev);
+	if (ret)
+		return ret;
+	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
+
+	/* Set up v4l2 device */
+	cam->v4l2_dev.mdev = &cam->media_dev;
+	ret = v4l2_device_register(dev, &cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
+		goto fail_media_unreg;
+	}
+	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
+
+	/* Initialize subdev */
+	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
+	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
+	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+				V4L2_SUBDEV_FL_HAS_EVENTS;
+	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
+		 "%s", dev_driver_string(dev));
+	v4l2_set_subdevdata(&cam->subdev, cam);
+
+	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev:%d\n", ret);
+		goto fail_clean_media_entiy;
+	}
+	dev_dbg(dev, "registered %s\n", cam->subdev.name);
+
+	/* Create video nodes and links */
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
+		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
+
+		node->id = node->desc.id;
+		ret = mtk_cam_video_register_device(cam, node);
+		if (ret)
+			goto fail_vdev_unreg;
+	}
+	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+	return 0;
+
+fail_vdev_unreg:
+	for (i--; i >= 0; i--)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+fail_clean_media_entiy:
+	media_entity_cleanup(&cam->subdev.entity);
+	v4l2_device_unregister(&cam->v4l2_dev);
+fail_media_unreg:
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return ret;
+}
+
+static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
+{
+	int i;
+
+	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
+		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
+
+	vb2_dma_contig_clear_max_seg_size(cam->dev);
+	v4l2_device_unregister_subdev(&cam->subdev);
+	v4l2_device_unregister(&cam->v4l2_dev);
+	media_entity_cleanup(&cam->subdev.entity);
+	media_device_unregister(&cam->media_dev);
+	media_device_cleanup(&cam->media_dev);
+
+	return 0;
+}
+
+static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
+				      struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
+		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
+		return -ENODEV;
+	}
+
+	cam->seninf = sd;
+	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
+
+	return 0;
+}
+
+static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
+					struct v4l2_subdev *sd,
+					struct v4l2_async_subdev *asd)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+
+	cam->seninf = NULL;
+	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
+}
+
+static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+	struct mtk_cam_dev *cam =
+		container_of(notifier, struct mtk_cam_dev, notifier);
+	struct device *dev = cam->dev;
+	int ret;
+
+	if (!cam->seninf) {
+		dev_err(dev, "No seninf subdev\n");
+		return -ENODEV;
+	}
+
+	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
+				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		dev_err(dev, "failed to create pad link %s %s err:%d\n",
+			cam->seninf->entity.name, cam->subdev.entity.name,
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
+	if (ret) {
+		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
+	.bound = mtk_cam_dev_notifier_bound,
+	.unbind = mtk_cam_dev_notifier_unbind,
+	.complete = mtk_cam_dev_notifier_complete,
+};
+
+static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
+{
+	struct device *dev = cam->dev;
+	int ret;
+
+	v4l2_async_notifier_init(&cam->notifier);
+	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
+		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
+	if (ret) {
+		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
+		return ret;
+	}
+
+	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
+	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
+	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
+	if (ret) {
+		dev_err(dev, "failed to register async notifier : %d\n", ret);
+		v4l2_async_notifier_cleanup(&cam->notifier);
+	}
+
+	return ret;
+}
+
+static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
+{
+	v4l2_async_notifier_unregister(&cam->notifier);
+	v4l2_async_notifier_cleanup(&cam->notifier);
+}
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
+	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
+	.vidioc_querycap = mtk_cam_vidioc_querycap,
+	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
+	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
+	.vidioc_reqbufs = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf = vb2_ioctl_querybuf,
+	.vidioc_qbuf = vb2_ioctl_qbuf,
+	.vidioc_dqbuf = vb2_ioctl_dqbuf,
+	.vidioc_streamon = vb2_ioctl_streamon,
+	.vidioc_streamoff = vb2_ioctl_streamoff,
+	.vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_format meta_fmts[] = {
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
+			.buffersize = 512 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_3A,
+			.buffersize = 1200 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_AF,
+			.buffersize = 640 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LCS,
+			.buffersize = 288 * SZ_1K,
+		},
+	},
+	{
+		.fmt.meta = {
+			.dataformat = V4L2_META_FMT_MTISP_LMV,
+			.buffersize = 256,
+		},
+	},
+};
+
+static const struct v4l2_format stream_out_fmts[] = {
+	/* This is a default image format */
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
+		},
+	},
+};
+
+static const struct v4l2_format bin_out_fmts[] = {
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
+		},
+	},
+	{
+		.fmt.pix_mp = {
+			.width = IMG_MAX_WIDTH,
+			.height = IMG_MAX_HEIGHT,
+			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
+		},
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc output_queues[] = {
+	{
+		.id = MTK_CAM_P1_META_IN_0,
+		.name = "meta input",
+		.cap = V4L2_CAP_META_OUTPUT,
+		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = true,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 0,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
+	},
+};
+
+static const struct
+mtk_cam_dev_node_desc capture_queues[] = {
+	{
+		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
+		.name = "main stream",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_IMGO,
+		.fmts = stream_out_fmts,
+		.num_fmts = ARRAY_SIZE(stream_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_PACKED_BIN_OUT,
+		.name = "packed out",
+		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
+		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+		.link_flags = 0,
+		.image = true,
+		.smem_alloc = false,
+		.dma_port = R_RRZO,
+		.fmts = bin_out_fmts,
+		.num_fmts = ARRAY_SIZE(bin_out_fmts),
+		.default_fmt_idx = 0,
+		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
+		.frmsizes = &(struct v4l2_frmsizeenum) {
+			.index = 0,
+			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+			.stepwise = {
+				.max_width = IMG_MAX_WIDTH,
+				.min_width = IMG_MIN_WIDTH,
+				.max_height = IMG_MAX_HEIGHT,
+				.min_height = IMG_MIN_HEIGHT,
+				.step_height = 1,
+				.step_width = 1,
+			},
+		},
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_0,
+		.name = "partial meta 0",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AAO | R_FLKO | R_PSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 1,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_1,
+		.name = "partial meta 1",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_AFO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 2,
+		.max_buf_count = 5,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_2,
+		.name = "partial meta 2",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LCSO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 3,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+	{
+		.id = MTK_CAM_P1_META_OUT_3,
+		.name = "partial meta 3",
+		.cap = V4L2_CAP_META_CAPTURE,
+		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+		.link_flags = 0,
+		.image = false,
+		.smem_alloc = false,
+		.dma_port = R_LMVO,
+		.fmts = meta_fmts,
+		.default_fmt_idx = 4,
+		.max_buf_count = 10,
+		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
+	},
+};
+
+/* The helper to configure the device context */
+static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
+{
+	unsigned int node_idx;
+	int i;
+
+	node_idx = 0;
+	/* Setup the output queue */
+	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = output_queues[i];
+
+	/* Setup the capture queue */
+	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
+		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
+}
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam)
+{
+	int ret;
+
+	cam->dev = &pdev->dev;
+	mtk_cam_dev_queue_setup(cam);
+
+	spin_lock_init(&cam->pending_job_lock);
+	spin_lock_init(&cam->running_job_lock);
+	INIT_LIST_HEAD(&cam->pending_job_list);
+	INIT_LIST_HEAD(&cam->running_job_list);
+	mutex_init(&cam->op_lock);
+
+	/* v4l2 sub-device registration */
+	ret = mtk_cam_v4l2_register(cam);
+	if (ret)
+		return ret;
+
+	ret = mtk_cam_v4l2_async_register(cam);
+	if (ret)
+		goto fail_v4l2_unreg;
+
+	return 0;
+
+fail_v4l2_unreg:
+	mutex_destroy(&cam->op_lock);
+	mtk_cam_v4l2_unregister(cam);
+
+	return ret;
+}
+
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
+{
+	mtk_cam_v4l2_async_unregister(cam);
+	mtk_cam_v4l2_unregister(cam);
+	mutex_destroy(&cam->op_lock);
+}
+
diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
new file mode 100644
index 000000000000..0a340a1e65ea
--- /dev/null
+++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_H__
+#define __MTK_CAM_H__
+
+#include <linux/device.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define IMG_MAX_WIDTH		5376
+#define IMG_MAX_HEIGHT		4032
+#define IMG_MIN_WIDTH		80
+#define IMG_MIN_HEIGHT		60
+
+/*
+ * ID enum value for struct mtk_cam_dev_node_desc:id
+ * or mtk_cam_video_device:id
+ */
+enum  {
+	MTK_CAM_P1_META_IN_0 = 0,
+	MTK_CAM_P1_MAIN_STREAM_OUT,
+	MTK_CAM_P1_PACKED_BIN_OUT,
+	MTK_CAM_P1_META_OUT_0,
+	MTK_CAM_P1_META_OUT_1,
+	MTK_CAM_P1_META_OUT_2,
+	MTK_CAM_P1_META_OUT_3,
+	MTK_CAM_P1_TOTAL_NODES
+};
+
+/* Supported image format list */
+#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
+#define MTK_CAM_IMG_FMT_BAYER8		0x2200
+#define MTK_CAM_IMG_FMT_BAYER10		0x2201
+#define MTK_CAM_IMG_FMT_BAYER12		0x2202
+#define MTK_CAM_IMG_FMT_BAYER14		0x2203
+#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
+#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
+#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
+#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
+
+/* Supported bayer pixel order */
+#define MTK_CAM_RAW_PXL_ID_B		0
+#define MTK_CAM_RAW_PXL_ID_GB		1
+#define MTK_CAM_RAW_PXL_ID_GR		2
+#define MTK_CAM_RAW_PXL_ID_R		3
+#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
+
+/*
+ * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
+ *
+ * @frame_seq_no: The frame sequence of frame in driver layer.
+ * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
+ *
+ */
+struct mtk_p1_frame_param {
+	unsigned int frame_seq_no;
+	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
+} __packed;
+
+/*
+ * struct mtk_cam_dev_request - MTK camera device request.
+ *
+ * @req: Embedded struct media request.
+ * @frame_params: The frame info. & address info. of enabled DMA nodes.
+ * @frame_work: work queue entry for frame transmission to SCP.
+ * @list: List entry of the object for @struct mtk_cam_dev:
+ *        pending_job_list or running_job_list.
+ * @timestamp: Start of frame timestamp in ns
+ *
+ */
+struct mtk_cam_dev_request {
+	struct media_request req;
+	struct mtk_p1_frame_param frame_params;
+	struct work_struct frame_work;
+	struct list_head list;
+	u64 timestamp;
+};
+
+/*
+ * struct mtk_cam_dev_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry of the object for @struct mtk_cam_video_device:
+ *        buf_list.
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ *            is only supported for meta input node.
+ * @node_id: The vidoe node id which this buffer belongs to.
+ *
+ */
+struct mtk_cam_dev_buffer {
+	struct vb2_v4l2_buffer vbb;
+	struct list_head list;
+	/* Intenal part */
+	dma_addr_t daddr;
+	dma_addr_t scp_addr;
+	unsigned int node_id;
+};
+
+/*
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops:  mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ *
+ */
+struct mtk_cam_dev_node_desc {
+	u8 id;
+	const char *name;
+	u32 cap;
+	u32 buf_type;
+	u32 dma_port;
+	u32 link_flags;
+	u8 smem_alloc:1;
+	u8 image:1;
+	u8 num_fmts;
+	u8 default_fmt_idx;
+	u8 max_buf_count;
+	const struct v4l2_ioctl_ops *ioctl_ops;
+	const struct v4l2_format *fmts;
+	const struct v4l2_frmsizeenum *frmsizes;
+};
+
+/*
+ * struct mtk_cam_video_device - Mediatek video device structure
+ *
+ * @id: Id for index of mtk_cam_dev:vdev_nodes array
+ * @enabled: Indicate the video device is enabled or not
+ * @desc: The node description of video device
+ * @vdev_fmt: The V4L2 format of video device
+ * @vdev_pad: The media pad graph object of video device
+ * @vbq: A videobuf queue of video device
+ * @vdev: The video device instance
+ * @vdev_lock: Serializes vb2 queue and video device operations
+ * @buf_list: List for enqueue buffers
+ * @buf_list_lock: Lock used to protect buffer list.
+ *
+ */
+struct mtk_cam_video_device {
+	unsigned int id;
+	unsigned int enabled;
+	struct mtk_cam_dev_node_desc desc;
+	struct v4l2_format vdev_fmt;
+	struct media_pad vdev_pad;
+	struct vb2_queue vbq;
+	struct video_device vdev;
+	/* Serializes vb2 queue and video device operations */
+	struct mutex vdev_lock;
+	struct list_head buf_list;
+	/* Lock used to protect buffer list */
+	spinlock_t buf_list_lock;
+};
+
+/*
+ * struct mtk_cam_dev - Mediatek camera device structure.
+ *
+ * @dev: Pointer to device.
+ * @smem_pdev: Pointer to shared memory device.
+ * @pipeline: Media pipeline information.
+ * @media_dev: Media device instance.
+ * @subdev: The V4L2 sub-device instance.
+ * @v4l2_dev: The V4L2 device driver instance.
+ * @notifier: The v4l2_device notifier data.
+ * @subdev_pads: Pointer to the number of media pads of this sub-device.
+ * @vdev_nodes: The array list of mtk_cam_video_device nodes.
+ * @seninf: Pointer to the seninf sub-device.
+ * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
+ * @streaming: Indicate the overall streaming status is on or off.
+ * @enabled_dmas: The enabled dma port information when streaming on.
+ * @enabled_count: Number of enabled video nodes
+ * @stream_count: Number of streaming video nodes
+ * @running_job_count: Nunber of running jobs in the HW driver.
+ * @pending_job_list: List to keep the media requests before en-queue into
+ *                    HW driver.
+ * @pending_job_lock: Protect the pending_job_list data & running_job_count.
+ * @running_job_list: List to keep the media requests after en-queue into
+ *                    HW driver.
+ * @running_job_lock: Protect the running_job_list data.
+ * @op_lock: Serializes driver's VB2 callback operations.
+ *
+ */
+struct mtk_cam_dev {
+	struct device *dev;
+	struct device *smem_dev;
+	struct media_pipeline pipeline;
+	struct media_device media_dev;
+	struct v4l2_subdev subdev;
+	struct v4l2_device v4l2_dev;
+	struct v4l2_async_notifier notifier;
+	struct media_pad *subdev_pads;
+	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
+	struct v4l2_subdev *seninf;
+	struct v4l2_subdev *sensor;
+	unsigned int streaming;
+	unsigned int enabled_dmas;
+	unsigned int enabled_count;
+	unsigned int stream_count;
+	unsigned int running_job_count;
+	struct list_head pending_job_list;
+	/* Protect the pending_job_list data */
+	spinlock_t pending_job_lock;
+	struct list_head running_job_list;
+	/* Protect the running_job_list data & running_job_count */
+	spinlock_t running_job_lock;
+	/* Serializes driver's VB2 callback operations */
+	struct mutex op_lock;
+};
+
+int mtk_cam_dev_init(struct platform_device *pdev,
+		     struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
+void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
+				   unsigned int frame_seq_no);
+void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
+				  unsigned int frame_seq_no);
+struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
+						unsigned int frame_seq_no);
+
+#endif /* __MTK_CAM_H__ */
-- 
2.18.0
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply related	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
  2019-12-19  5:49     ` Jungo Lin
  (?)
@ 2020-01-07 14:10       ` Hans Verkuil
  -1 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-07 14:10 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

On 12/19/19 6:49 AM, Jungo Lin wrote:
> For Camera AR(Augmented Reality) application requires camera timestamps
> to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> sources.
> 
> The boottime timestamp is identical to monotonic timestamp,
> except it also includes any time that the system is suspended.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - No change.
> ---
>  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
>  include/uapi/linux/videodev2.h          |  2 ++
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..f45bfce7fddd 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -662,13 +662,22 @@ Buffer Flags
>        - 0x00002000
>        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
>  	clock. To access the same clock outside V4L2, use
> -	:c:func:`clock_gettime`.
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.

IDs -> ID

>      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
>  
>        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
>        - 0x00004000
>        - The CAPTURE buffer timestamp has been taken from the corresponding
>  	OUTPUT buffer. This flag applies only to mem2mem devices.
> +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:

You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.

> +
> +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> +      - 0x00008000
> +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> +	clock. To access the same clock outside V4L2, use
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.

IDs -> ID

> +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> +	the system is suspended.
>      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
>  
>        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 04481c717fee..74ef9472e702 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000

This should be 0x00006000.

(flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
source, so these timestamp defines are values, not bitmasks.

However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
a userspace decision, and locking a driver to one of these two options seems
wrong to me.

Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
the BOOTTIME timestamp. This requires a simple helper function that returns either
ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.

It's definitely more work (although it can be limited to drivers that use vb2),
but much more useful.

Regards,

	Hans

> +
>  /* Timestamp sources. */
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> 


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2020-01-07 14:10       ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-07 14:10 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

On 12/19/19 6:49 AM, Jungo Lin wrote:
> For Camera AR(Augmented Reality) application requires camera timestamps
> to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> sources.
> 
> The boottime timestamp is identical to monotonic timestamp,
> except it also includes any time that the system is suspended.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - No change.
> ---
>  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
>  include/uapi/linux/videodev2.h          |  2 ++
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..f45bfce7fddd 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -662,13 +662,22 @@ Buffer Flags
>        - 0x00002000
>        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
>  	clock. To access the same clock outside V4L2, use
> -	:c:func:`clock_gettime`.
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.

IDs -> ID

>      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
>  
>        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
>        - 0x00004000
>        - The CAPTURE buffer timestamp has been taken from the corresponding
>  	OUTPUT buffer. This flag applies only to mem2mem devices.
> +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:

You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.

> +
> +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> +      - 0x00008000
> +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> +	clock. To access the same clock outside V4L2, use
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.

IDs -> ID

> +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> +	the system is suspended.
>      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
>  
>        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 04481c717fee..74ef9472e702 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000

This should be 0x00006000.

(flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
source, so these timestamp defines are values, not bitmasks.

However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
a userspace decision, and locking a driver to one of these two options seems
wrong to me.

Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
the BOOTTIME timestamp. This requires a simple helper function that returns either
ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.

It's definitely more work (although it can be limited to drivers that use vb2),
but much more useful.

Regards,

	Hans

> +
>  /* Timestamp sources. */
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> 


_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2020-01-07 14:10       ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-07 14:10 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

On 12/19/19 6:49 AM, Jungo Lin wrote:
> For Camera AR(Augmented Reality) application requires camera timestamps
> to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> sources.
> 
> The boottime timestamp is identical to monotonic timestamp,
> except it also includes any time that the system is suspended.
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - No change.
> ---
>  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
>  include/uapi/linux/videodev2.h          |  2 ++
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..f45bfce7fddd 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -662,13 +662,22 @@ Buffer Flags
>        - 0x00002000
>        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
>  	clock. To access the same clock outside V4L2, use
> -	:c:func:`clock_gettime`.
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.

IDs -> ID

>      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
>  
>        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
>        - 0x00004000
>        - The CAPTURE buffer timestamp has been taken from the corresponding
>  	OUTPUT buffer. This flag applies only to mem2mem devices.
> +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:

You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.

> +
> +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> +      - 0x00008000
> +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> +	clock. To access the same clock outside V4L2, use
> +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.

IDs -> ID

> +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> +	the system is suspended.
>      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
>  
>        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 04481c717fee..74ef9472e702 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000

This should be 0x00006000.

(flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
source, so these timestamp defines are values, not bitmasks.

However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
a userspace decision, and locking a driver to one of these two options seems
wrong to me.

Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
the BOOTTIME timestamp. This requires a simple helper function that returns either
ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.

It's definitely more work (although it can be limited to drivers that use vb2),
but much more useful.

Regards,

	Hans

> +
>  /* Timestamp sources. */
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
>  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
       [not found]       ` <e833b88ba74945c495a102c98cd54725@mtkmbs07n1.mediatek.inc>
@ 2020-01-10  9:59         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-10  9:59 UTC (permalink / raw)
  To: hverkuil-cisco; +Cc: linux-media

Hi Hans:

Appreciate your comments on this patch.

> From: Hans Verkuil [mailto:hverkuil-cisco@xs4all.nl]
> Sent: Tuesday, January 07, 2020 10:11 PM
> To: jungo.lin@mediatek.com; tfiga@chromium.org; laurent.pinchart@ideasonboard.com; matthias.bgg@gmail.com; mchehab@kernel.org
> Cc: linux-media@vger.kernel.org; linux-mediatek@lists.infradead.org; linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org; srv_heupstream@mediatek.com; ddavenport@chromium.org; robh@kernel.org; Sean.Cheng@mediatek.com; sj.huang@mediatek.com; Frederic.Chen@mediatek.com; Jerry-ch.Chen@mediatek.com; Frankie.Chiu@mediatek.com; ryan.yu@mediatek.com; Rynn.Wu@mediatek.com; yuzhao@chromium.org; zwisler@chromium.org; shik@chromium.org; suleiman@chromium.org
> Subject: Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
> 
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > For Camera AR(Augmented Reality) application requires camera
> > timestamps to be reported with CLOCK_BOOTTIME to sync timestamp with
> > other sensor sources.
> >
> > The boottime timestamp is identical to monotonic timestamp, except it
> > also includes any time that the system is suspended.
> >
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - No change.
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
> >  include/uapi/linux/videodev2.h          |  2 ++
> >  2 files changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst
> > b/Documentation/media/uapi/v4l/buffer.rst
> > index 9149b57728e5..f45bfce7fddd 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -662,13 +662,22 @@ Buffer Flags
> >        - 0x00002000
> >        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
> >  clock. To access the same clock outside V4L2, use
> > -:c:func:`clock_gettime`.
> > +:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
> 
> IDs -> ID
> 

Ok, fix in next version.

> >      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
> >
> >        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
> >        - 0x00004000
> >        - The CAPTURE buffer timestamp has been taken from the corresponding
> >  OUTPUT buffer. This flag applies only to mem2mem devices.
> > +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
> 
> You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.
> 

Ok, fix this typo in next version.

> > +
> > +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> > +      - 0x00008000
> > +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> > +clock. To access the same clock outside V4L2, use
> > +:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
> 
> IDs -> ID
> 

Ditto.

> > +Identical to CLOCK_MONOTONIC, except it also includes any time that
> > +the system is suspended.
> >      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
> >
> >        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK`` diff --git
> > a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 04481c717fee..74ef9472e702 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
> >  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN0x00000000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC0x00002000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_COPY0x00004000
> > +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME0x00008000
> 
> This should be 0x00006000.
> 
> (flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp source, so these timestamp defines are values, not bitmasks.
> 
> However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really a userspace decision, and locking a driver to one of these two options seems wrong to me.
> 
> Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing the buffer and that indicates that instead of the MONOTONIC timestamp, it should return the BOOTTIME timestamp. This requires a simple helper function that returns either ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.
> 
> It's definitely more work (although it can be limited to drivers that use vb2), but much more useful.
> 
> Regards,
> 
> Hans
> 

Agree.
We will add new V4L2_BUF_FLAG_USE_BOOTTIME flag (0x00006000.) to replace
this V4L2_BUF_FLAG_TIMESTAMP_BOOTIME flag for better usage.

Sincerely

Jungo

> > +
> >  /* Timestamp sources. */
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK0x00070000
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF0x00000000
> >




^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
  2020-01-07 14:10       ` Hans Verkuil
  (?)
@ 2020-01-10 10:08         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-10 10:08 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: tfiga, laurent.pinchart, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

Hi Hans:

Appreciate your comments on this patch.

On Tue, 2020-01-07 at 15:10 +0100, Hans Verkuil wrote:
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > For Camera AR(Augmented Reality) application requires camera timestamps
> > to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> > sources.
> > 
> > The boottime timestamp is identical to monotonic timestamp,
> > except it also includes any time that the system is suspended.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - No change.
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
> >  include/uapi/linux/videodev2.h          |  2 ++
> >  2 files changed, 12 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> > index 9149b57728e5..f45bfce7fddd 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -662,13 +662,22 @@ Buffer Flags
> >        - 0x00002000
> >        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
> >  	clock. To access the same clock outside V4L2, use
> > -	:c:func:`clock_gettime`.
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
> 
> IDs -> ID
> 

Ok, fix in next version.

> >      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
> >  
> >        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
> >        - 0x00004000
> >        - The CAPTURE buffer timestamp has been taken from the corresponding
> >  	OUTPUT buffer. This flag applies only to mem2mem devices.
> > +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
> 
> You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.
> 

Ok, fix this typo in next version.

> > +
> > +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> > +      - 0x00008000
> > +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> > +	clock. To access the same clock outside V4L2, use
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
> 
> IDs -> ID
> 

Ditto.

> > +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> > +	the system is suspended.
> >      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
> >  
> >        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 04481c717fee..74ef9472e702 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
> >  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> > +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
> 
> This should be 0x00006000.
> 
> (flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
> source, so these timestamp defines are values, not bitmasks.
> 
> However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
> a userspace decision, and locking a driver to one of these two options seems
> wrong to me.
> 
> Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
> the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
> the BOOTTIME timestamp. This requires a simple helper function that returns either
> ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.
> 
> It's definitely more work (although it can be limited to drivers that use vb2),
> but much more useful.
> 
> Regards,
> 
> 	Hans
> 

Agree.
We will add new V4L2_BUF_FLAG_USE_BOOTTIME flag (0x00006000.) to replace
this V4L2_BUF_FLAG_TIMESTAMP_BOOTIME flag for better usage.

> > +
> >  /* Timestamp sources. */
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> > 
> 

Sincerely

Jungo

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2020-01-10 10:08         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-10 10:08 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi Hans:

Appreciate your comments on this patch.

On Tue, 2020-01-07 at 15:10 +0100, Hans Verkuil wrote:
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > For Camera AR(Augmented Reality) application requires camera timestamps
> > to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> > sources.
> > 
> > The boottime timestamp is identical to monotonic timestamp,
> > except it also includes any time that the system is suspended.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - No change.
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
> >  include/uapi/linux/videodev2.h          |  2 ++
> >  2 files changed, 12 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> > index 9149b57728e5..f45bfce7fddd 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -662,13 +662,22 @@ Buffer Flags
> >        - 0x00002000
> >        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
> >  	clock. To access the same clock outside V4L2, use
> > -	:c:func:`clock_gettime`.
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
> 
> IDs -> ID
> 

Ok, fix in next version.

> >      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
> >  
> >        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
> >        - 0x00004000
> >        - The CAPTURE buffer timestamp has been taken from the corresponding
> >  	OUTPUT buffer. This flag applies only to mem2mem devices.
> > +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
> 
> You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.
> 

Ok, fix this typo in next version.

> > +
> > +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> > +      - 0x00008000
> > +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> > +	clock. To access the same clock outside V4L2, use
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
> 
> IDs -> ID
> 

Ditto.

> > +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> > +	the system is suspended.
> >      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
> >  
> >        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 04481c717fee..74ef9472e702 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
> >  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> > +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
> 
> This should be 0x00006000.
> 
> (flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
> source, so these timestamp defines are values, not bitmasks.
> 
> However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
> a userspace decision, and locking a driver to one of these two options seems
> wrong to me.
> 
> Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
> the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
> the BOOTTIME timestamp. This requires a simple helper function that returns either
> ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.
> 
> It's definitely more work (although it can be limited to drivers that use vb2),
> but much more useful.
> 
> Regards,
> 
> 	Hans
> 

Agree.
We will add new V4L2_BUF_FLAG_USE_BOOTTIME flag (0x00006000.) to replace
this V4L2_BUF_FLAG_TIMESTAMP_BOOTIME flag for better usage.

> > +
> >  /* Timestamp sources. */
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> > 
> 

Sincerely

Jungo
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 3/5] media: videodev2.h: Add new boottime timestamp type
@ 2020-01-10 10:08         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-10 10:08 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi Hans:

Appreciate your comments on this patch.

On Tue, 2020-01-07 at 15:10 +0100, Hans Verkuil wrote:
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > For Camera AR(Augmented Reality) application requires camera timestamps
> > to be reported with CLOCK_BOOTTIME to sync timestamp with other sensor
> > sources.
> > 
> > The boottime timestamp is identical to monotonic timestamp,
> > except it also includes any time that the system is suspended.
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - No change.
> > ---
> >  Documentation/media/uapi/v4l/buffer.rst | 11 ++++++++++-
> >  include/uapi/linux/videodev2.h          |  2 ++
> >  2 files changed, 12 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> > index 9149b57728e5..f45bfce7fddd 100644
> > --- a/Documentation/media/uapi/v4l/buffer.rst
> > +++ b/Documentation/media/uapi/v4l/buffer.rst
> > @@ -662,13 +662,22 @@ Buffer Flags
> >        - 0x00002000
> >        - The buffer timestamp has been taken from the ``CLOCK_MONOTONIC``
> >  	clock. To access the same clock outside V4L2, use
> > -	:c:func:`clock_gettime`.
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_MONOTONIC``.
> 
> IDs -> ID
> 

Ok, fix in next version.

> >      * .. _`V4L2-BUF-FLAG-TIMESTAMP-COPY`:
> >  
> >        - ``V4L2_BUF_FLAG_TIMESTAMP_COPY``
> >        - 0x00004000
> >        - The CAPTURE buffer timestamp has been taken from the corresponding
> >  	OUTPUT buffer. This flag applies only to mem2mem devices.
> > +    * .. _`V4L2_BUF_FLAG_TIMESTAMP_BOOTIME`:
> 
> You mistyped BOOTTIME as BOOTIME in a lot of places. Please check.
> 

Ok, fix this typo in next version.

> > +
> > +      - ``V4L2_BUF_FLAG_TIMESTAMP_BOOTIME``
> > +      - 0x00008000
> > +      - The buffer timestamp has been taken from the ``CLOCK_BOOTTIME``
> > +	clock. To access the same clock outside V4L2, use
> > +	:c:func:`clock_gettime` using clock IDs ``CLOCK_BOOTTIME``.
> 
> IDs -> ID
> 

Ditto.

> > +	Identical to CLOCK_MONOTONIC, except it also includes any time that
> > +	the system is suspended.
> >      * .. _`V4L2-BUF-FLAG-TSTAMP-SRC-MASK`:
> >  
> >        - ``V4L2_BUF_FLAG_TSTAMP_SRC_MASK``
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 04481c717fee..74ef9472e702 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -1060,6 +1060,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
> >  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x00000000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x00002000
> >  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x00004000
> > +#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME		0x00008000
> 
> This should be 0x00006000.
> 
> (flags & V4L2_BUF_FLAG_TIMESTAMP_MASK) is a value that determines the timestamp
> source, so these timestamp defines are values, not bitmasks.
> 
> However, I don't like your approach. Whether to use MONOTONIC or BOOTTIME is really
> a userspace decision, and locking a driver to one of these two options seems
> wrong to me.
> 
> Instead add new V4L2_BUF_FLAG_USE_BOOTTIME flag that userspace can set when queuing
> the buffer and that indicates that instead of the MONOTONIC timestamp, it should return
> the BOOTTIME timestamp. This requires a simple helper function that returns either
> ktime_get_ns or ktime_get_boottime_ns based on the vb2_v4l2_buffer flags field.
> 
> It's definitely more work (although it can be limited to drivers that use vb2),
> but much more useful.
> 
> Regards,
> 
> 	Hans
> 

Agree.
We will add new V4L2_BUF_FLAG_USE_BOOTTIME flag (0x00006000.) to replace
this V4L2_BUF_FLAG_TIMESTAMP_BOOTIME flag for better usage.

> > +
> >  /* Timestamp sources. */
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_MASK		0x00070000
> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_EOF		0x00000000
> > 
> 

Sincerely

Jungo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-12-19  5:49     ` Jungo Lin
  (?)
@ 2020-01-23 13:59       ` Hans Verkuil
  -1 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-23 13:59 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Pi-Hsun Shih

Hi Jungo,

On 12/19/19 6:49 AM, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>  - Correct auto suspend timer value for suspend/resume issue
>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>  - Fix KE due to no sen-inf sub-device
> ---
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  9 files changed, 3377 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

<snip>

> +static void isp_tx_frame_worker(struct work_struct *work)
> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}

<snip>

> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}

<snip>

> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;
> +	u32 scp_addr;
> +} __packed;

<snip>

> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> +}
> +

<snip>

> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;

So if I understand this correctly, to set the ISP frame parameters userspace
provides an array of pointers to other memory areas that are magically created
somewhere and containing magic, undocumented data.

I know you said that this is 'not ready to be reviewed yet', but I just wanted
to mention that this is of course not acceptable and needs to be replaced with
a documented metadata structure that userspace can pass in the metadata buffer.

Just ignore this email if you were already planning on doing that. I just wanted
to make sure that it is clear that the current approach won't fly.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-01-23 13:59       ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-23 13:59 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hi Jungo,

On 12/19/19 6:49 AM, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>  - Correct auto suspend timer value for suspend/resume issue
>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>  - Fix KE due to no sen-inf sub-device
> ---
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  9 files changed, 3377 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

<snip>

> +static void isp_tx_frame_worker(struct work_struct *work)
> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}

<snip>

> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}

<snip>

> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;
> +	u32 scp_addr;
> +} __packed;

<snip>

> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> +}
> +

<snip>

> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;

So if I understand this correctly, to set the ISP frame parameters userspace
provides an array of pointers to other memory areas that are magically created
somewhere and containing magic, undocumented data.

I know you said that this is 'not ready to be reviewed yet', but I just wanted
to mention that this is of course not acceptable and needs to be replaced with
a documented metadata structure that userspace can pass in the metadata buffer.

Just ignore this email if you were already planning on doing that. I just wanted
to make sure that it is clear that the current approach won't fly.

Regards,

	Hans

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-01-23 13:59       ` Hans Verkuil
  0 siblings, 0 replies; 388+ messages in thread
From: Hans Verkuil @ 2020-01-23 13:59 UTC (permalink / raw)
  To: Jungo Lin, tfiga, laurent.pinchart, matthias.bgg, mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hi Jungo,

On 12/19/19 6:49 AM, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>  - Correct auto suspend timer value for suspend/resume issue
>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>  - Fix KE due to no sen-inf sub-device
> ---
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  9 files changed, 3377 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h

<snip>

> +static void isp_tx_frame_worker(struct work_struct *work)
> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}

<snip>

> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}

<snip>

> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;
> +	u32 scp_addr;
> +} __packed;

<snip>

> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> +}
> +

<snip>

> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;

So if I understand this correctly, to set the ISP frame parameters userspace
provides an array of pointers to other memory areas that are magically created
somewhere and containing magic, undocumented data.

I know you said that this is 'not ready to be reviewed yet', but I just wanted
to mention that this is of course not acceptable and needs to be replaced with
a documented metadata structure that userspace can pass in the metadata buffer.

Just ignore this email if you were already planning on doing that. I just wanted
to make sure that it is clear that the current approach won't fly.

Regards,

	Hans

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2020-01-23 13:59       ` Hans Verkuil
  (?)
@ 2020-01-28  2:13         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-28  2:13 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: tfiga, laurent.pinchart, matthias.bgg, mchehab, shik, devicetree,
	Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi, Hans:

On Thu, 2020-01-23 at 14:59 +0100, Hans Verkuil wrote:
> Hi Jungo,
> 
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >  - Correct auto suspend timer value for suspend/resume issue
> >  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >  - Fix KE due to no sen-inf sub-device
> > ---
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >  9 files changed, 3377 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> <snip>
> 
> > +static void isp_tx_frame_worker(struct work_struct *work)
> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> 
> <snip>
> 
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> 
> <snip>
> 
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> > +	u32 scp_addr;
> > +} __packed;
> 
> <snip>
> 
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> > +}
> > +
> 
> <snip>
> 
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> 
> So if I understand this correctly, to set the ISP frame parameters userspace
> provides an array of pointers to other memory areas that are magically created
> somewhere and containing magic, undocumented data.
> 
> I know you said that this is 'not ready to be reviewed yet', but I just wanted
> to mention that this is of course not acceptable and needs to be replaced with
> a documented metadata structure that userspace can pass in the metadata buffer.
> 
> Just ignore this email if you were already planning on doing that. I just wanted
> to make sure that it is clear that the current approach won't fly.
> 
> Regards,
> 
> 	Hans
> 

Thanks for your comment.

Firstly, I think I miss meta data types definition in this series.
https://patchwork.kernel.org/patch/11126055/
include/uapi/linux/videodev2.h
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /*
AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF
histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /*
Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /*
Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /*
ISP tuning parameters */
We will correct this missing error in next patch set.

Secondly, we are working on the documented meta-data structures for
these meta nodes, especially on 4L2_META_FMT_MTISP_PARAMS which is used
for tuning parameters  from user space.

Sincerely

Jungo

> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-01-28  2:13         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-28  2:13 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree, sj.huang,
	yuzhao, linux-mediatek, Pi-Hsun Shih, matthias.bgg, mchehab,
	linux-arm-kernel, Sean.Cheng, srv_heupstream, shik, tfiga,
	zwisler, ddavenport

Hi, Hans:

On Thu, 2020-01-23 at 14:59 +0100, Hans Verkuil wrote:
> Hi Jungo,
> 
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >  - Correct auto suspend timer value for suspend/resume issue
> >  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >  - Fix KE due to no sen-inf sub-device
> > ---
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >  9 files changed, 3377 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> <snip>
> 
> > +static void isp_tx_frame_worker(struct work_struct *work)
> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> 
> <snip>
> 
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> 
> <snip>
> 
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> > +	u32 scp_addr;
> > +} __packed;
> 
> <snip>
> 
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> > +}
> > +
> 
> <snip>
> 
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> 
> So if I understand this correctly, to set the ISP frame parameters userspace
> provides an array of pointers to other memory areas that are magically created
> somewhere and containing magic, undocumented data.
> 
> I know you said that this is 'not ready to be reviewed yet', but I just wanted
> to mention that this is of course not acceptable and needs to be replaced with
> a documented metadata structure that userspace can pass in the metadata buffer.
> 
> Just ignore this email if you were already planning on doing that. I just wanted
> to make sure that it is clear that the current approach won't fly.
> 
> Regards,
> 
> 	Hans
> 

Thanks for your comment.

Firstly, I think I miss meta data types definition in this series.
https://patchwork.kernel.org/patch/11126055/
include/uapi/linux/videodev2.h
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /*
AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF
histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /*
Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /*
Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /*
ISP tuning parameters */
We will correct this missing error in next patch set.

Secondly, we are working on the documented meta-data structures for
these meta nodes, especially on 4L2_META_FMT_MTISP_PARAMS which is used
for tuning parameters  from user space.

Sincerely

Jungo

> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-01-28  2:13         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-01-28  2:13 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree, sj.huang,
	yuzhao, linux-mediatek, Pi-Hsun Shih, matthias.bgg, mchehab,
	linux-arm-kernel, Sean.Cheng, srv_heupstream, shik, tfiga,
	zwisler, ddavenport

Hi, Hans:

On Thu, 2020-01-23 at 14:59 +0100, Hans Verkuil wrote:
> Hi Jungo,
> 
> On 12/19/19 6:49 AM, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >  - Correct auto suspend timer value for suspend/resume issue
> >  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >  - Fix KE due to no sen-inf sub-device
> > ---
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >  9 files changed, 3377 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> <snip>
> 
> > +static void isp_tx_frame_worker(struct work_struct *work)
> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> 
> <snip>
> 
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> 
> <snip>
> 
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> > +	u32 scp_addr;
> > +} __packed;
> 
> <snip>
> 
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> > +}
> > +
> 
> <snip>
> 
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> 
> So if I understand this correctly, to set the ISP frame parameters userspace
> provides an array of pointers to other memory areas that are magically created
> somewhere and containing magic, undocumented data.
> 
> I know you said that this is 'not ready to be reviewed yet', but I just wanted
> to mention that this is of course not acceptable and needs to be replaced with
> a documented metadata structure that userspace can pass in the metadata buffer.
> 
> Just ignore this email if you were already planning on doing that. I just wanted
> to make sure that it is clear that the current approach won't fly.
> 
> Regards,
> 
> 	Hans
> 

Thanks for your comment.

Firstly, I think I miss meta data types definition in this series.
https://patchwork.kernel.org/patch/11126055/
include/uapi/linux/videodev2.h
+#define V4L2_META_FMT_MTISP_3A    v4l2_fourcc('M', 'T', 'f', 'a') /*
AE/AWB histogram */
+#define V4L2_META_FMT_MTISP_AF    v4l2_fourcc('M', 'T', 'f', 'f') /* AF
histogram */
+#define V4L2_META_FMT_MTISP_LCS   v4l2_fourcc('M', 'T', 'f', 'c') /*
Local contrast enhanced statistics */
+#define V4L2_META_FMT_MTISP_LMV   v4l2_fourcc('M', 'T', 'f', 'm') /*
Local motion vector histogram */
+#define V4L2_META_FMT_MTISP_PARAMS v4l2_fourcc('M', 'T', 'f', 'p') /*
ISP tuning parameters */
We will correct this missing error in next patch set.

Secondly, we are working on the documented meta-data structures for
these meta nodes, especially on 4L2_META_FMT_MTISP_PARAMS which is used
for tuning parameters  from user space.

Sincerely

Jungo

> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
  2019-12-19  5:49   ` Jungo Lin
  (?)
@ 2020-03-31 15:34     ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

Hi Jungo,

I was taking a look at this patchset, please see my comments below.

On 12/19/19 3:49 AM, Jungo Lin wrote:
> Hello,
> 
> This patch series adding the driver for Pass 1 (P1) unit in
> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> camera features of CrOS.
> 
> Pass 1 unit processes image signal from sensor devices and accepts the
> tuning parameters to adjust the image quality. It performs optical
> black correction, defect pixel correction, W/IR imbalance correction
> and lens shading correction for RAW processing.
> 
> The driver is implemented with V4L2 and media controller framework so
> we have the following entities to describe the ISP pass 1 path.
> 
> (The current metadata interface used in meta input and partial meta
> nodes is only a temporary solution to kick off the driver development
> and is not ready to be reviewed yet.)
> 
> 1. meta input (output video device): connect to ISP P1 sub device.
> It accepts the tuning buffer from user.
> 
> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> main stream and packed out video devices. When processing an image,
> Pass 1 hardware supports multiple output images with different sizes
> and formats so it needs two capture video devices ("main stream" and
> "packed out") to return the image data to the user.
> 
> 3. main stream (capture video device): return the processed image data
> which is used in capture scenario.
> 
> 4. packed out (capture video device): return the processed image data
> which is used in preview scenario.
> 
> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> 
> 6. partial meta 1 (capture video device): return the AF statistics.
> 
> 7. partial meta 2 (capture video device): return the local contrast
>    enhanced statistics.
> 
> 8. partial meta 3 (capture video device): return the local motion
>    vector statistics.
> 
> The overall patches of the series is:
> 
> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>   Moreover, implement standard V4L2 video driver that utilizes
>   V4L2 and media framework APIs. Communicate with co-process via SCP
>   communication to compose ISP registers in the firmware.
> 
> Here is ISP P1 media topology:
> It is included the main/sub sensor, sen-inf sub-devices and len device
> which are implemented in below patch[1][2][3][4]:

I would be nice if you could provide a branch with those applied.

> 
> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> patch sets.
> 
> /usr/bin/media-ctl -p -d /dev/media2
> 
> Media controller API version 4.19.89
> 
> Media device information
> ------------------------
> driver          mtk-cam-p1
> model           mtk-cam-p1
> serial          
> bus info        platform:1a000000.camisp
> hw revision     0x0
> driver version  4.19.89
> 
> Device topology
> - entity 1: mtk-cam-p1 (12 pads, 8 links)

If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
Is this correct?

So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.

>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
> 	pad0: Sink
> 		<- "mtk-cam-p1 meta input":0 []

I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.

> 	pad1: Source
> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]

Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?

> 	pad2: Source
> 		-> "mtk-cam-p1 packed out":0 []

Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?

> 	pad3: Source
> 		-> "mtk-cam-p1 partial meta 0":0 []
> 	pad4: Source
> 		-> "mtk-cam-p1 partial meta 1":0 []
> 	pad5: Source
> 		-> "mtk-cam-p1 partial meta 2":0 []
> 	pad6: Source
> 		-> "mtk-cam-p1 partial meta 3":0 []

Shouldn't those links be [ENABLED,IMMUTABLE] ?

It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
"mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?

I also would prefer to remove blank spaces.

And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.

> 	pad7: Source
> 	pad8: Source
> 	pad9: Source
> 	pad10: Source

Why source pads that are not connected to anything? (I guess I need to check the last patch better).

Regards,
Helen

> 	pad11: Sink
> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> 
> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video2
> 	pad0: Source
> 		-> "mtk-cam-p1":0 []
> 
> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video3
> 	pad0: Sink
> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> 
> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video4
> 	pad0: Sink
> 		<- "mtk-cam-p1":2 []
> 
> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video5
> 	pad0: Sink
> 		<- "mtk-cam-p1":3 []
> 
> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video6
> 	pad0: Sink
> 		<- "mtk-cam-p1":4 []
> 
> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video7
> 	pad0: Sink
> 		<- "mtk-cam-p1":5 []
> 
> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video8
> 	pad0: Sink
> 		<- "mtk-cam-p1":6 []
> 
> - entity 56: 1a040000.seninf (12 pads, 3 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev1
> 	pad0: Sink
> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> 		<- "ov8856 2-0010":0 [ENABLED]
> 	pad1: Sink
> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> 		<- "ov02a10 4-003d":0 []
> 	pad2: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad3: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad4: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> 	pad5: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad6: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad7: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad8: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad9: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad10: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad11: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 
> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev2
> 	pad0: Source
> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> 		-> "1a040000.seninf":0 [ENABLED]
> 
> - entity 73: dw9768 2-000c (0 pad, 0 link)
>              type V4L2 subdev subtype Lens flags 0
>              device node name /dev/v4l-subdev3
> 
> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev4
> 	pad0: Source
> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
> 		-> "1a040000.seninf":1 []
> 
> ===========
> = history =
> ===========
> 
> version 6:
>  - Add port node description in the dt-binding document and device tree
>    by Tomasz Figa.
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>    version.
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>  - Correct auto suspend timer value for suspend/resume issue.
>  - Increase IPI guard timer to 1 second to avoid false alarm command
>    timeout event.
>  - Fix KE due to no sen-inf sub-device.
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation.
>  - Add rst documents for Mediatek meta formats.
>  - New meta buffer structure design & re-factoring.
> 
> version 5:
>  - Fixed Rob's comment on dt-binding format
>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>    and new timestamp type in driver
>  - Fix buffer en-queue timing issue in v4
>  - Remove default link_notify callback function in mtk_cam_media_ops
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 4:
>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>    patch[4]
>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>  - Extend Mediatek proprietary image formats to support bayer order
>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 3:
>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>    reserved memory region. (Rob Herring)
>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>  - Revise ISP Pass1 Kconfig
>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>  - Fix kernel warning messages when running v4l2_compliance test
>  - Move AFO buffer enqueue & de-queue from request API to non-request
>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>    declaration (Hans Verkuil)
>    Split GET_BIN_INFO control into two controls to get width & height
>    in-dependently (Hans Verkuil)
>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Fix ISP de-initialize timing KE issue
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> 
> Todo:
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
> 
> version 2:
>  - Add 3A enhancement feature which includes:
>    Separates 3A pipeline out of frame basis to improve
>    AE/AWB (exposure and white balance) performance.
>    Add 2 SCP sub-commands for 3A meta buffers.
>  - Add new child device to manage P1 shared memory between P1 HW unit
>    and co-processor.
>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>  - Revised document wording for dt-bindings documents & dts information.
>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>  - mtk_cam-dev.h / mtk_cam-dev.c
>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>    or add comments.
>    Revised buffer size for LMVO & LCSO.
>    Fix pixel format utility function.
>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>  - mtk_cam-v4l2-util.c
>    Refactoring V4L2 async mechanism with seninf driver only
>    Refactoring CIO (Connection IO) implementation with active sensor
>    Revised stream on function for 3A enhancement feature
>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Add meta buffer index register definitions
>    Add meta DMA configuration function.
>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>    Add isp_setup_scp_rproc function to get RPC handle
>    Add mtk_cam_reserved_memory_init for shared memory management
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Add new meta strictures for 3A enhancement feature
>    Add new IPI command utility function for 3A enhancement feature
>    Enhance isp_composer_dma_sg_init function flow
>    Shorten overall IPI command structure size
>    Remove scp_state state checking
>    Improve code readability
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>    memory region.
> 
> TODOs:
>  - 3A enhancement feature bug fixing
> 
> version 1:
>  - Revised driver sources based on Tomasz's comments including
>    part1/2/3/4 in RFC V0 patch.
>  - Remove DMA cache mechanism.
>    Support two new video devices (LCSO/LMVO) for advance camera
>    features.
>  - Fixed v4l2-compliance test failure items.
>  - Add private controls for Mediatek camera middle-ware.
>  - Replace VPU driver's APIs with new SCP driver interface for
>    co-processor communication.
>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>    commands RX handling.
>  - Fix internal bugs.
> 
> TODOs:
>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>    for shared memory management.
>  - Revised file names.
>  - Support non frame-sync AFO/AAO DMA buffers
> 
> version 0:
> - Initial submission
> 
> ==================
>  Dependent patch set
> ==================
> 
> Camera ISP P1 driver depends on seninf driver, SCP driver.
> The patches are listed as following:
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/11145845/
> 
> [2]. media: ov8856: Add YAML binding and sensor mode support
> https://patchwork.kernel.org/cover/11220785/
> 
> [3]. media: i2c: Add support for OV02A10 sensor
> https://patchwork.kernel.org/cover/11284779/
> 
> [4]. media: i2c: add support for DW9768 VCM driver
> https://patchwork.kernel.org/cover/11132299/
> 
> [5]. Add support for mt8183 SCP
> https://patchwork.kernel.org/cover/11239065/
> 
> [6]. MT8183 IOMMU SUPPORT
> https://patchwork.kernel.org/cover/11112765/
> 
> ==================
>  Compliance test
> ==================
> 
> The v4l2-compliance is built with the below lastest patch.
> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> 
> Note 1.
> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> 
> Note 2.
> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> 
> Note 3.
> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> 
> /usr/bin/v4l2-compliance -m /dev/media2
> 
> v4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           :
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media1 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video25:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.67
> 	Capabilities     : 0x8c200000
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video25 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composiv4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media2 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video2:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x8c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video2 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video3:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000016
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000014 (20)
> 	Name             : mtk-cam-p1 main stream
> 	Function         : V4L2 I/O
> 	Pad 0x01000015   : 0: Sink
> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video3 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video4:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300001c
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000001a (26)
> 	Name             : mtk-cam-p1 packed out
> 	Function         : V4L2 I/O
> 	Pad 0x0100001b   : 0: Sink
> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video4 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video5:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000022
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000020 (32)
> 	Name             : mtk-cam-p1 partial meta 0
> 	Function         : V4L2 I/O
> 	Pad 0x01000021   : 0: Sink
> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video5 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video6:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000028
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000026 (38)
> 	Name             : mtk-cam-p1 partial meta 1
> 	Function         : V4L2 I/O
> 	Pad 0x01000027   : 0: Sink
> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video6 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video7:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300002e
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000002c (44)
> 	Name             : mtk-cam-p1 partial meta 2
> 	Function         : V4L2 I/O
> 	Pad 0x0100002d   : 0: Sink
> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video7 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video8:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000034
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000032 (50)
> 	Name             : mtk-cam-p1 partial meta 3
> 	Function         : V4L2 I/O
> 	Pad 0x01000033   : 0: Sink
> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video8 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000050
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000001 (1)
> 	Name             : mtk-cam-p1
> 	Function         : Video Pixel Formatter
> 	Pad 0x01000002   : 0: Sink
> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> 	Pad 0x01000003   : 1: Source
> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> 	Pad 0x01000004   : 2: Source
> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> 	Pad 0x01000005   : 3: Source
> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> 	Pad 0x01000006   : 4: Source
> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> 	Pad 0x01000007   : 5: Source
> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> 	Pad 0x01000008   : 6: Source
> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> 	Pad 0x01000009   : 7: Source
> 	Pad 0x0100000a   : 8: Source
> 	Pad 0x0100000b   : 9: Source
> 	Pad 0x0100000c   : 10: Source
> 	Pad 0x0100000d   : 11: Sink
> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev0 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000052
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000038 (56)
> 	Name             : 1a040000.seninf
> 	Function         : Video Interface Bridge
> 	Pad 0x01000039   : 0: Sink
> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> 	Pad 0x0100003a   : 1: Sink
> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> 	Pad 0x0100003b   : 2: Sink
> 	Pad 0x0100003c   : 3: Sink
> 	Pad 0x0100003d   : 4: Source
> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 	Pad 0x0100003e   : 5: Source
> 	Pad 0x0100003f   : 6: Source
> 	Pad 0x01000040   : 7: Source
> 	Pad 0x01000041   : 8: Source
> 	Pad 0x01000042   : 9: Source
> 	Pad 0x01000043   : 10: Source
> 	Pad 0x01000044   : 11: Source
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev1 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000054
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000045 (69)
> 	Name             : ov8856 2-0010
> 	Function         : Camera Sensor
> 	Pad 0x01000046   : 0: Source
> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev2 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 11 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000056
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000049 (73)
> 	Name             : dw9768 2-000c
> 	Function         : Lens Controller
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev3 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000058
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x0000004a (74)
> 	Name             : ov02a10 4-003d
> 	Function         : Camera Sensor
> 	Pad 0x0100004b   : 0: Source
> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev4 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> 	test VIDIOC_G/S_CTRL: FAIL
> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 10 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> 
> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> 
> 
> Jungo Lin (5):
>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>   media: videodev2.h: Add new boottime timestamp type
>   media: platform: Add Mediatek ISP P1 image & meta formats
>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> 
>  .../bindings/media/mediatek,camisp.txt        |   83 +
>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>  include/uapi/linux/videodev2.h                |   41 +
>  24 files changed, 4226 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-03-31 15:34     ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Jungo,

I was taking a look at this patchset, please see my comments below.

On 12/19/19 3:49 AM, Jungo Lin wrote:
> Hello,
> 
> This patch series adding the driver for Pass 1 (P1) unit in
> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> camera features of CrOS.
> 
> Pass 1 unit processes image signal from sensor devices and accepts the
> tuning parameters to adjust the image quality. It performs optical
> black correction, defect pixel correction, W/IR imbalance correction
> and lens shading correction for RAW processing.
> 
> The driver is implemented with V4L2 and media controller framework so
> we have the following entities to describe the ISP pass 1 path.
> 
> (The current metadata interface used in meta input and partial meta
> nodes is only a temporary solution to kick off the driver development
> and is not ready to be reviewed yet.)
> 
> 1. meta input (output video device): connect to ISP P1 sub device.
> It accepts the tuning buffer from user.
> 
> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> main stream and packed out video devices. When processing an image,
> Pass 1 hardware supports multiple output images with different sizes
> and formats so it needs two capture video devices ("main stream" and
> "packed out") to return the image data to the user.
> 
> 3. main stream (capture video device): return the processed image data
> which is used in capture scenario.
> 
> 4. packed out (capture video device): return the processed image data
> which is used in preview scenario.
> 
> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> 
> 6. partial meta 1 (capture video device): return the AF statistics.
> 
> 7. partial meta 2 (capture video device): return the local contrast
>    enhanced statistics.
> 
> 8. partial meta 3 (capture video device): return the local motion
>    vector statistics.
> 
> The overall patches of the series is:
> 
> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>   Moreover, implement standard V4L2 video driver that utilizes
>   V4L2 and media framework APIs. Communicate with co-process via SCP
>   communication to compose ISP registers in the firmware.
> 
> Here is ISP P1 media topology:
> It is included the main/sub sensor, sen-inf sub-devices and len device
> which are implemented in below patch[1][2][3][4]:

I would be nice if you could provide a branch with those applied.

> 
> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> patch sets.
> 
> /usr/bin/media-ctl -p -d /dev/media2
> 
> Media controller API version 4.19.89
> 
> Media device information
> ------------------------
> driver          mtk-cam-p1
> model           mtk-cam-p1
> serial          
> bus info        platform:1a000000.camisp
> hw revision     0x0
> driver version  4.19.89
> 
> Device topology
> - entity 1: mtk-cam-p1 (12 pads, 8 links)

If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
Is this correct?

So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.

>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
> 	pad0: Sink
> 		<- "mtk-cam-p1 meta input":0 []

I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.

> 	pad1: Source
> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]

Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?

> 	pad2: Source
> 		-> "mtk-cam-p1 packed out":0 []

Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?

> 	pad3: Source
> 		-> "mtk-cam-p1 partial meta 0":0 []
> 	pad4: Source
> 		-> "mtk-cam-p1 partial meta 1":0 []
> 	pad5: Source
> 		-> "mtk-cam-p1 partial meta 2":0 []
> 	pad6: Source
> 		-> "mtk-cam-p1 partial meta 3":0 []

Shouldn't those links be [ENABLED,IMMUTABLE] ?

It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
"mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?

I also would prefer to remove blank spaces.

And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.

> 	pad7: Source
> 	pad8: Source
> 	pad9: Source
> 	pad10: Source

Why source pads that are not connected to anything? (I guess I need to check the last patch better).

Regards,
Helen

> 	pad11: Sink
> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> 
> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video2
> 	pad0: Source
> 		-> "mtk-cam-p1":0 []
> 
> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video3
> 	pad0: Sink
> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> 
> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video4
> 	pad0: Sink
> 		<- "mtk-cam-p1":2 []
> 
> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video5
> 	pad0: Sink
> 		<- "mtk-cam-p1":3 []
> 
> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video6
> 	pad0: Sink
> 		<- "mtk-cam-p1":4 []
> 
> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video7
> 	pad0: Sink
> 		<- "mtk-cam-p1":5 []
> 
> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video8
> 	pad0: Sink
> 		<- "mtk-cam-p1":6 []
> 
> - entity 56: 1a040000.seninf (12 pads, 3 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev1
> 	pad0: Sink
> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> 		<- "ov8856 2-0010":0 [ENABLED]
> 	pad1: Sink
> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> 		<- "ov02a10 4-003d":0 []
> 	pad2: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad3: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad4: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> 	pad5: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad6: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad7: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad8: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad9: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad10: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad11: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 
> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev2
> 	pad0: Source
> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> 		-> "1a040000.seninf":0 [ENABLED]
> 
> - entity 73: dw9768 2-000c (0 pad, 0 link)
>              type V4L2 subdev subtype Lens flags 0
>              device node name /dev/v4l-subdev3
> 
> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev4
> 	pad0: Source
> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
> 		-> "1a040000.seninf":1 []
> 
> ===========
> = history =
> ===========
> 
> version 6:
>  - Add port node description in the dt-binding document and device tree
>    by Tomasz Figa.
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>    version.
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>  - Correct auto suspend timer value for suspend/resume issue.
>  - Increase IPI guard timer to 1 second to avoid false alarm command
>    timeout event.
>  - Fix KE due to no sen-inf sub-device.
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation.
>  - Add rst documents for Mediatek meta formats.
>  - New meta buffer structure design & re-factoring.
> 
> version 5:
>  - Fixed Rob's comment on dt-binding format
>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>    and new timestamp type in driver
>  - Fix buffer en-queue timing issue in v4
>  - Remove default link_notify callback function in mtk_cam_media_ops
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 4:
>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>    patch[4]
>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>  - Extend Mediatek proprietary image formats to support bayer order
>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 3:
>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>    reserved memory region. (Rob Herring)
>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>  - Revise ISP Pass1 Kconfig
>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>  - Fix kernel warning messages when running v4l2_compliance test
>  - Move AFO buffer enqueue & de-queue from request API to non-request
>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>    declaration (Hans Verkuil)
>    Split GET_BIN_INFO control into two controls to get width & height
>    in-dependently (Hans Verkuil)
>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Fix ISP de-initialize timing KE issue
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> 
> Todo:
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
> 
> version 2:
>  - Add 3A enhancement feature which includes:
>    Separates 3A pipeline out of frame basis to improve
>    AE/AWB (exposure and white balance) performance.
>    Add 2 SCP sub-commands for 3A meta buffers.
>  - Add new child device to manage P1 shared memory between P1 HW unit
>    and co-processor.
>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>  - Revised document wording for dt-bindings documents & dts information.
>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>  - mtk_cam-dev.h / mtk_cam-dev.c
>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>    or add comments.
>    Revised buffer size for LMVO & LCSO.
>    Fix pixel format utility function.
>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>  - mtk_cam-v4l2-util.c
>    Refactoring V4L2 async mechanism with seninf driver only
>    Refactoring CIO (Connection IO) implementation with active sensor
>    Revised stream on function for 3A enhancement feature
>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Add meta buffer index register definitions
>    Add meta DMA configuration function.
>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>    Add isp_setup_scp_rproc function to get RPC handle
>    Add mtk_cam_reserved_memory_init for shared memory management
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Add new meta strictures for 3A enhancement feature
>    Add new IPI command utility function for 3A enhancement feature
>    Enhance isp_composer_dma_sg_init function flow
>    Shorten overall IPI command structure size
>    Remove scp_state state checking
>    Improve code readability
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>    memory region.
> 
> TODOs:
>  - 3A enhancement feature bug fixing
> 
> version 1:
>  - Revised driver sources based on Tomasz's comments including
>    part1/2/3/4 in RFC V0 patch.
>  - Remove DMA cache mechanism.
>    Support two new video devices (LCSO/LMVO) for advance camera
>    features.
>  - Fixed v4l2-compliance test failure items.
>  - Add private controls for Mediatek camera middle-ware.
>  - Replace VPU driver's APIs with new SCP driver interface for
>    co-processor communication.
>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>    commands RX handling.
>  - Fix internal bugs.
> 
> TODOs:
>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>    for shared memory management.
>  - Revised file names.
>  - Support non frame-sync AFO/AAO DMA buffers
> 
> version 0:
> - Initial submission
> 
> ==================
>  Dependent patch set
> ==================
> 
> Camera ISP P1 driver depends on seninf driver, SCP driver.
> The patches are listed as following:
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/11145845/
> 
> [2]. media: ov8856: Add YAML binding and sensor mode support
> https://patchwork.kernel.org/cover/11220785/
> 
> [3]. media: i2c: Add support for OV02A10 sensor
> https://patchwork.kernel.org/cover/11284779/
> 
> [4]. media: i2c: add support for DW9768 VCM driver
> https://patchwork.kernel.org/cover/11132299/
> 
> [5]. Add support for mt8183 SCP
> https://patchwork.kernel.org/cover/11239065/
> 
> [6]. MT8183 IOMMU SUPPORT
> https://patchwork.kernel.org/cover/11112765/
> 
> ==================
>  Compliance test
> ==================
> 
> The v4l2-compliance is built with the below lastest patch.
> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> 
> Note 1.
> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> 
> Note 2.
> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> 
> Note 3.
> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> 
> /usr/bin/v4l2-compliance -m /dev/media2
> 
> v4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           :
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media1 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video25:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.67
> 	Capabilities     : 0x8c200000
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video25 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composiv4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media2 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video2:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x8c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video2 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video3:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000016
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000014 (20)
> 	Name             : mtk-cam-p1 main stream
> 	Function         : V4L2 I/O
> 	Pad 0x01000015   : 0: Sink
> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video3 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video4:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300001c
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000001a (26)
> 	Name             : mtk-cam-p1 packed out
> 	Function         : V4L2 I/O
> 	Pad 0x0100001b   : 0: Sink
> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video4 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video5:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000022
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000020 (32)
> 	Name             : mtk-cam-p1 partial meta 0
> 	Function         : V4L2 I/O
> 	Pad 0x01000021   : 0: Sink
> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video5 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video6:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000028
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000026 (38)
> 	Name             : mtk-cam-p1 partial meta 1
> 	Function         : V4L2 I/O
> 	Pad 0x01000027   : 0: Sink
> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video6 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video7:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300002e
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000002c (44)
> 	Name             : mtk-cam-p1 partial meta 2
> 	Function         : V4L2 I/O
> 	Pad 0x0100002d   : 0: Sink
> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video7 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video8:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000034
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000032 (50)
> 	Name             : mtk-cam-p1 partial meta 3
> 	Function         : V4L2 I/O
> 	Pad 0x01000033   : 0: Sink
> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video8 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000050
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000001 (1)
> 	Name             : mtk-cam-p1
> 	Function         : Video Pixel Formatter
> 	Pad 0x01000002   : 0: Sink
> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> 	Pad 0x01000003   : 1: Source
> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> 	Pad 0x01000004   : 2: Source
> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> 	Pad 0x01000005   : 3: Source
> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> 	Pad 0x01000006   : 4: Source
> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> 	Pad 0x01000007   : 5: Source
> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> 	Pad 0x01000008   : 6: Source
> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> 	Pad 0x01000009   : 7: Source
> 	Pad 0x0100000a   : 8: Source
> 	Pad 0x0100000b   : 9: Source
> 	Pad 0x0100000c   : 10: Source
> 	Pad 0x0100000d   : 11: Sink
> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev0 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000052
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000038 (56)
> 	Name             : 1a040000.seninf
> 	Function         : Video Interface Bridge
> 	Pad 0x01000039   : 0: Sink
> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> 	Pad 0x0100003a   : 1: Sink
> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> 	Pad 0x0100003b   : 2: Sink
> 	Pad 0x0100003c   : 3: Sink
> 	Pad 0x0100003d   : 4: Source
> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 	Pad 0x0100003e   : 5: Source
> 	Pad 0x0100003f   : 6: Source
> 	Pad 0x01000040   : 7: Source
> 	Pad 0x01000041   : 8: Source
> 	Pad 0x01000042   : 9: Source
> 	Pad 0x01000043   : 10: Source
> 	Pad 0x01000044   : 11: Source
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev1 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000054
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000045 (69)
> 	Name             : ov8856 2-0010
> 	Function         : Camera Sensor
> 	Pad 0x01000046   : 0: Source
> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev2 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 11 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000056
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000049 (73)
> 	Name             : dw9768 2-000c
> 	Function         : Lens Controller
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev3 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000058
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x0000004a (74)
> 	Name             : ov02a10 4-003d
> 	Function         : Camera Sensor
> 	Pad 0x0100004b   : 0: Source
> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev4 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> 	test VIDIOC_G/S_CTRL: FAIL
> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 10 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> 
> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> 
> 
> Jungo Lin (5):
>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>   media: videodev2.h: Add new boottime timestamp type
>   media: platform: Add Mediatek ISP P1 image & meta formats
>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> 
>  .../bindings/media/mediatek,camisp.txt        |   83 +
>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>  include/uapi/linux/videodev2.h                |   41 +
>  24 files changed, 4226 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-03-31 15:34     ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Jungo,

I was taking a look at this patchset, please see my comments below.

On 12/19/19 3:49 AM, Jungo Lin wrote:
> Hello,
> 
> This patch series adding the driver for Pass 1 (P1) unit in
> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> camera features of CrOS.
> 
> Pass 1 unit processes image signal from sensor devices and accepts the
> tuning parameters to adjust the image quality. It performs optical
> black correction, defect pixel correction, W/IR imbalance correction
> and lens shading correction for RAW processing.
> 
> The driver is implemented with V4L2 and media controller framework so
> we have the following entities to describe the ISP pass 1 path.
> 
> (The current metadata interface used in meta input and partial meta
> nodes is only a temporary solution to kick off the driver development
> and is not ready to be reviewed yet.)
> 
> 1. meta input (output video device): connect to ISP P1 sub device.
> It accepts the tuning buffer from user.
> 
> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> main stream and packed out video devices. When processing an image,
> Pass 1 hardware supports multiple output images with different sizes
> and formats so it needs two capture video devices ("main stream" and
> "packed out") to return the image data to the user.
> 
> 3. main stream (capture video device): return the processed image data
> which is used in capture scenario.
> 
> 4. packed out (capture video device): return the processed image data
> which is used in preview scenario.
> 
> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> 
> 6. partial meta 1 (capture video device): return the AF statistics.
> 
> 7. partial meta 2 (capture video device): return the local contrast
>    enhanced statistics.
> 
> 8. partial meta 3 (capture video device): return the local motion
>    vector statistics.
> 
> The overall patches of the series is:
> 
> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>   Moreover, implement standard V4L2 video driver that utilizes
>   V4L2 and media framework APIs. Communicate with co-process via SCP
>   communication to compose ISP registers in the firmware.
> 
> Here is ISP P1 media topology:
> It is included the main/sub sensor, sen-inf sub-devices and len device
> which are implemented in below patch[1][2][3][4]:

I would be nice if you could provide a branch with those applied.

> 
> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> patch sets.
> 
> /usr/bin/media-ctl -p -d /dev/media2
> 
> Media controller API version 4.19.89
> 
> Media device information
> ------------------------
> driver          mtk-cam-p1
> model           mtk-cam-p1
> serial          
> bus info        platform:1a000000.camisp
> hw revision     0x0
> driver version  4.19.89
> 
> Device topology
> - entity 1: mtk-cam-p1 (12 pads, 8 links)

If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
Is this correct?

So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.

>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
> 	pad0: Sink
> 		<- "mtk-cam-p1 meta input":0 []

I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.

> 	pad1: Source
> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]

Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?

> 	pad2: Source
> 		-> "mtk-cam-p1 packed out":0 []

Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?

> 	pad3: Source
> 		-> "mtk-cam-p1 partial meta 0":0 []
> 	pad4: Source
> 		-> "mtk-cam-p1 partial meta 1":0 []
> 	pad5: Source
> 		-> "mtk-cam-p1 partial meta 2":0 []
> 	pad6: Source
> 		-> "mtk-cam-p1 partial meta 3":0 []

Shouldn't those links be [ENABLED,IMMUTABLE] ?

It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
"mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?

I also would prefer to remove blank spaces.

And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.

> 	pad7: Source
> 	pad8: Source
> 	pad9: Source
> 	pad10: Source

Why source pads that are not connected to anything? (I guess I need to check the last patch better).

Regards,
Helen

> 	pad11: Sink
> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> 
> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video2
> 	pad0: Source
> 		-> "mtk-cam-p1":0 []
> 
> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video3
> 	pad0: Sink
> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> 
> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video4
> 	pad0: Sink
> 		<- "mtk-cam-p1":2 []
> 
> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video5
> 	pad0: Sink
> 		<- "mtk-cam-p1":3 []
> 
> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video6
> 	pad0: Sink
> 		<- "mtk-cam-p1":4 []
> 
> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video7
> 	pad0: Sink
> 		<- "mtk-cam-p1":5 []
> 
> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video8
> 	pad0: Sink
> 		<- "mtk-cam-p1":6 []
> 
> - entity 56: 1a040000.seninf (12 pads, 3 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev1
> 	pad0: Sink
> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> 		<- "ov8856 2-0010":0 [ENABLED]
> 	pad1: Sink
> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> 		<- "ov02a10 4-003d":0 []
> 	pad2: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad3: Sink
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad4: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> 	pad5: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad6: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad7: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad8: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad9: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad10: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 	pad11: Source
> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> 
> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev2
> 	pad0: Source
> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> 		-> "1a040000.seninf":0 [ENABLED]
> 
> - entity 73: dw9768 2-000c (0 pad, 0 link)
>              type V4L2 subdev subtype Lens flags 0
>              device node name /dev/v4l-subdev3
> 
> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev4
> 	pad0: Source
> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
> 		-> "1a040000.seninf":1 []
> 
> ===========
> = history =
> ===========
> 
> version 6:
>  - Add port node description in the dt-binding document and device tree
>    by Tomasz Figa.
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>    version.
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>  - Correct auto suspend timer value for suspend/resume issue.
>  - Increase IPI guard timer to 1 second to avoid false alarm command
>    timeout event.
>  - Fix KE due to no sen-inf sub-device.
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation.
>  - Add rst documents for Mediatek meta formats.
>  - New meta buffer structure design & re-factoring.
> 
> version 5:
>  - Fixed Rob's comment on dt-binding format
>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>    and new timestamp type in driver
>  - Fix buffer en-queue timing issue in v4
>  - Remove default link_notify callback function in mtk_cam_media_ops
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 4:
>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>    patch[4]
>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>  - Extend Mediatek proprietary image formats to support bayer order
>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> 
> Todo:
>  - vb2_ops's buf_request_complete callback function implementation
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>  - Align and pack IPI command structures for EC ROM size shrink
> 
> version 3:
>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>    reserved memory region. (Rob Herring)
>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>  - Revise ISP Pass1 Kconfig
>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>  - Fix kernel warning messages when running v4l2_compliance test
>  - Move AFO buffer enqueue & de-queue from request API to non-request
>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>    declaration (Hans Verkuil)
>    Split GET_BIN_INFO control into two controls to get width & height
>    in-dependently (Hans Verkuil)
>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Fix Drew's comments which are addressed in v2 patch
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>    Fix ISP de-initialize timing KE issue
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> 
> Todo:
>  - Add rst documents for Mediatek meta formats
>  - New meta buffer structure design & re-factoring
> 
> version 2:
>  - Add 3A enhancement feature which includes:
>    Separates 3A pipeline out of frame basis to improve
>    AE/AWB (exposure and white balance) performance.
>    Add 2 SCP sub-commands for 3A meta buffers.
>  - Add new child device to manage P1 shared memory between P1 HW unit
>    and co-processor.
>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>  - Revised document wording for dt-bindings documents & dts information.
>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>  - mtk_cam-dev.h / mtk_cam-dev.c
>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>    or add comments.
>    Revised buffer size for LMVO & LCSO.
>    Fix pixel format utility function.
>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>  - mtk_cam-v4l2-util.c
>    Refactoring V4L2 async mechanism with seninf driver only
>    Refactoring CIO (Connection IO) implementation with active sensor
>    Revised stream on function for 3A enhancement feature
>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>    Add meta buffer index register definitions
>    Add meta DMA configuration function.
>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>    Add isp_setup_scp_rproc function to get RPC handle
>    Add mtk_cam_reserved_memory_init for shared memory management
>  - mtk_cam-scp.h / mtk_cam-scp.c
>    Add new meta strictures for 3A enhancement feature
>    Add new IPI command utility function for 3A enhancement feature
>    Enhance isp_composer_dma_sg_init function flow
>    Shorten overall IPI command structure size
>    Remove scp_state state checking
>    Improve code readability
>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>    memory region.
> 
> TODOs:
>  - 3A enhancement feature bug fixing
> 
> version 1:
>  - Revised driver sources based on Tomasz's comments including
>    part1/2/3/4 in RFC V0 patch.
>  - Remove DMA cache mechanism.
>    Support two new video devices (LCSO/LMVO) for advance camera
>    features.
>  - Fixed v4l2-compliance test failure items.
>  - Add private controls for Mediatek camera middle-ware.
>  - Replace VPU driver's APIs with new SCP driver interface for
>    co-processor communication.
>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>    commands RX handling.
>  - Fix internal bugs.
> 
> TODOs:
>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>    for shared memory management.
>  - Revised file names.
>  - Support non frame-sync AFO/AAO DMA buffers
> 
> version 0:
> - Initial submission
> 
> ==================
>  Dependent patch set
> ==================
> 
> Camera ISP P1 driver depends on seninf driver, SCP driver.
> The patches are listed as following:
> 
> [1]. media: support Mediatek sensor interface driver
> https://patchwork.kernel.org/cover/11145845/
> 
> [2]. media: ov8856: Add YAML binding and sensor mode support
> https://patchwork.kernel.org/cover/11220785/
> 
> [3]. media: i2c: Add support for OV02A10 sensor
> https://patchwork.kernel.org/cover/11284779/
> 
> [4]. media: i2c: add support for DW9768 VCM driver
> https://patchwork.kernel.org/cover/11132299/
> 
> [5]. Add support for mt8183 SCP
> https://patchwork.kernel.org/cover/11239065/
> 
> [6]. MT8183 IOMMU SUPPORT
> https://patchwork.kernel.org/cover/11112765/
> 
> ==================
>  Compliance test
> ==================
> 
> The v4l2-compliance is built with the below lastest patch.
> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> 
> Note 1.
> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> 
> Note 2.
> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> 
> Note 3.
> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> 
> /usr/bin/v4l2-compliance -m /dev/media2
> 
> v4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           :
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media1 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video25:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.67
> 	Capabilities     : 0x8c200000
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.67
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.67
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video25 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composiv4l2-compliance SHA: not available, 32 bits
> 
> Compliance test for mtk-cam-p1 device /dev/media2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 
> Allow for multiple opens:
> 	test second /dev/media2 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video2:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x8c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x0c200000
> 		Metadata Output
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000010
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000e (14)
> 	Name             : mtk-cam-p1 meta input
> 	Function         : V4L2 I/O
> 	Pad 0x0100000f   : 0: Source
> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video2 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video3:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000016
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000014 (20)
> 	Name             : mtk-cam-p1 main stream
> 	Function         : V4L2 I/O
> 	Pad 0x01000015   : 0: Sink
> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video3 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video4:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04201000
> 		Video Capture Multiplanar
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300001c
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000001a (26)
> 	Name             : mtk-cam-p1 packed out
> 	Function         : V4L2 I/O
> 	Pad 0x0100001b   : 0: Sink
> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video4 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video5:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000022
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000020 (32)
> 	Name             : mtk-cam-p1 partial meta 0
> 	Function         : V4L2 I/O
> 	Pad 0x01000021   : 0: Sink
> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video5 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video6:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000028
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000026 (38)
> 	Name             : mtk-cam-p1 partial meta 1
> 	Function         : V4L2 I/O
> 	Pad 0x01000027   : 0: Sink
> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video6 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video7:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x0300002e
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000002c (44)
> 	Name             : mtk-cam-p1 partial meta 2
> 	Function         : V4L2 I/O
> 	Pad 0x0100002d   : 0: Sink
> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video7 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/video8:
> 
> Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Card type        : mtk-cam-p1
> 	Bus info         : platform:1a000000.camisp
> 	Driver version   : 4.19.89
> 	Capabilities     : 0x84a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04a00000
> 		Metadata Capture
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000034
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000032 (50)
> 	Name             : mtk-cam-p1 partial meta 3
> 	Function         : V4L2 I/O
> 	Pad 0x01000033   : 0: Sink
> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video8 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK
> 
> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000050
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000001 (1)
> 	Name             : mtk-cam-p1
> 	Function         : Video Pixel Formatter
> 	Pad 0x01000002   : 0: Sink
> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> 	Pad 0x01000003   : 1: Source
> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> 	Pad 0x01000004   : 2: Source
> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> 	Pad 0x01000005   : 3: Source
> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> 	Pad 0x01000006   : 4: Source
> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> 	Pad 0x01000007   : 5: Source
> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> 	Pad 0x01000008   : 6: Source
> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> 	Pad 0x01000009   : 7: Source
> 	Pad 0x0100000a   : 8: Source
> 	Pad 0x0100000b   : 9: Source
> 	Pad 0x0100000c   : 10: Source
> 	Pad 0x0100000d   : 11: Sink
> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev0 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000052
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000038 (56)
> 	Name             : 1a040000.seninf
> 	Function         : Video Interface Bridge
> 	Pad 0x01000039   : 0: Sink
> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> 	Pad 0x0100003a   : 1: Sink
> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> 	Pad 0x0100003b   : 2: Sink
> 	Pad 0x0100003c   : 3: Sink
> 	Pad 0x0100003d   : 4: Source
> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> 	Pad 0x0100003e   : 5: Source
> 	Pad 0x0100003f   : 6: Source
> 	Pad 0x01000040   : 7: Source
> 	Pad 0x01000041   : 8: Source
> 	Pad 0x01000042   : 9: Source
> 	Pad 0x01000043   : 10: Source
> 	Pad 0x01000044   : 11: Source
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev1 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 1):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 2):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Sink Pad 3):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 4):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 5):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 6):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 7):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 8):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 9):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 10):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 11):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000054
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000045 (69)
> 	Name             : ov8856 2-0010
> 	Function         : Camera Sensor
> 	Pad 0x01000046   : 0: Source
> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev2 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 11 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000056
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000049 (73)
> 	Name             : dw9768 2-000c
> 	Function         : Lens Controller
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev3 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> 
> Media Driver Info:
> 	Driver name      : mtk-cam-p1
> 	Model            : mtk-cam-p1
> 	Serial           : 
> 	Bus info         : platform:1a000000.camisp
> 	Media version    : 4.19.89
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 4.19.89
> Interface Info:
> 	ID               : 0x03000058
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x0000004a (74)
> 	Name             : ov02a10 4-003d
> 	Function         : Camera Sensor
> 	Pad 0x0100004b   : 0: Source
> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev4 open: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device ioctls (Source Pad 0):
> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> 	test VIDIOC_G/S_CTRL: FAIL
> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 10 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> 
> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> 
> 
> Jungo Lin (5):
>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>   media: videodev2.h: Add new boottime timestamp type
>   media: platform: Add Mediatek ISP P1 image & meta formats
>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> 
>  .../bindings/media/mediatek,camisp.txt        |   83 +
>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>  include/uapi/linux/videodev2.h                |   41 +
>  24 files changed, 4226 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2019-12-19  5:49     ` Jungo Lin
  (?)
@ 2020-03-31 15:34       ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman

Hi Jungo,

On 12/19/19 3:49 AM, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Reviewed-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Add port node description in the dt-binding document.
> ---
>  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++

It would be really nice to convert this to yaml.

For reference: https://lwn.net/Articles/771621/

Regards,
Helen

>  1 file changed, 83 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..a85f37c0b87d
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,83 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp: The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- port: child port node corresponding to the data input, in accordance with
> +  the video interface bindings defined in
> +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> +  node must contain at least one endpoint.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp";
> +		reg = <0 0x1a000000 0 0x1000>,
> +				<0 0x1a003000 0 0x1000>,
> +				<0 0x1a004000 0 0x2000>,
> +				<0 0x1a006000 0 0x2000>,
> +				<0 0x1a008000 0 0x2000>;
> +		reg-names = "cam_sys",
> +				"cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +		clocks = <&camsys CLK_CAM_CAM>,
> +				<&camsys CLK_CAM_CAMTG>;
> +		clock-names = "camsys_cam_cgpdn",
> +				"camsys_camtg_cgpdn";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		mediatek,scp = <&scp>;
> +
> +		port {
> +			camisp_endpoint: endpoint {
> +				remote-endpoint = <&seninf_camisp_endpoint>;
> +			};
> +		};
> +	};
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2020-03-31 15:34       ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Jungo,

On 12/19/19 3:49 AM, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Reviewed-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Add port node description in the dt-binding document.
> ---
>  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++

It would be really nice to convert this to yaml.

For reference: https://lwn.net/Articles/771621/

Regards,
Helen

>  1 file changed, 83 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..a85f37c0b87d
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,83 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp: The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- port: child port node corresponding to the data input, in accordance with
> +  the video interface bindings defined in
> +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> +  node must contain at least one endpoint.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp";
> +		reg = <0 0x1a000000 0 0x1000>,
> +				<0 0x1a003000 0 0x1000>,
> +				<0 0x1a004000 0 0x2000>,
> +				<0 0x1a006000 0 0x2000>,
> +				<0 0x1a008000 0 0x2000>;
> +		reg-names = "cam_sys",
> +				"cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +		clocks = <&camsys CLK_CAM_CAM>,
> +				<&camsys CLK_CAM_CAMTG>;
> +		clock-names = "camsys_cam_cgpdn",
> +				"camsys_camtg_cgpdn";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		mediatek,scp = <&scp>;
> +
> +		port {
> +			camisp_endpoint: endpoint {
> +				remote-endpoint = <&seninf_camisp_endpoint>;
> +			};
> +		};
> +	};
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2020-03-31 15:34       ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Jungo,

On 12/19/19 3:49 AM, Jungo Lin wrote:
> This patch adds DT binding document for the Pass 1 (P1) unit
> in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> data out from the sensor interface, applies ISP image effects
> from tuning data and outputs the image data or statistics data to DRAM.
> 
> Reviewed-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Add port node description in the dt-binding document.
> ---
>  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++

It would be really nice to convert this to yaml.

For reference: https://lwn.net/Articles/771621/

Regards,
Helen

>  1 file changed, 83 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> new file mode 100644
> index 000000000000..a85f37c0b87d
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> @@ -0,0 +1,83 @@
> +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> +
> +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> +from the sensor interface, applies ISP effects from tuning data and outputs
> +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> +the ability to output two different resolutions frames at the same time to
> +increase the performance of the camera application.
> +
> +Required properties:
> +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> +- reg: Physical base address of the camera function block register and
> +  length of memory mapped region. Must contain an entry for each entry
> +  in reg-names.
> +- reg-names: Must include the following entries:
> +  "cam_sys": Camera base function block
> +  "cam_uni": Camera UNI function block
> +  "cam_a": Camera ISP P1 hardware unit A
> +  "cam_b": Camera ISP P1 hardware unit B
> +  "cam_c": Camera ISP P1 hardware unit C
> +- interrupts: Must contain an entry for each entry in interrupt-names.
> +- interrupt-names : Must include the following entries:
> +  "cam_uni": Camera UNI interrupt
> +  "cam_a": Camera unit A interrupt
> +  "cam_b": Camera unit B interrupt
> +  "cam_c": Camera unit C interrupt
> +- iommus: Shall point to the respective IOMMU block with master port
> +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> +  for details.
> +- clocks: A list of phandle and clock specifier pairs as listed
> +  in clock-names property, see
> +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> +  for details.
> +- power-domains: a phandle to the power domain, see
> +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> +- mediatek,scp: The node of system control processor (SCP), see
> +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> +- port: child port node corresponding to the data input, in accordance with
> +  the video interface bindings defined in
> +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> +  node must contain at least one endpoint.
> +
> +Example:
> +SoC specific DT entry:
> +
> +	camisp: camisp@1a000000 {
> +		compatible = "mediatek,mt8183-camisp";
> +		reg = <0 0x1a000000 0 0x1000>,
> +				<0 0x1a003000 0 0x1000>,
> +				<0 0x1a004000 0 0x2000>,
> +				<0 0x1a006000 0 0x2000>,
> +				<0 0x1a008000 0 0x2000>;
> +		reg-names = "cam_sys",
> +				"cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> +		interrupt-names = "cam_uni",
> +				"cam_a",
> +				"cam_b",
> +				"cam_c";
> +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +		clocks = <&camsys CLK_CAM_CAM>,
> +				<&camsys CLK_CAM_CAMTG>;
> +		clock-names = "camsys_cam_cgpdn",
> +				"camsys_camtg_cgpdn";
> +		mediatek,larb = <&larb3>,
> +				<&larb6>;
> +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> +		mediatek,scp = <&scp>;
> +
> +		port {
> +			camisp_endpoint: endpoint {
> +				remote-endpoint = <&seninf_camisp_endpoint>;
> +			};
> +		};
> +	};
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-12-19  5:49     ` Jungo Lin
@ 2020-03-31 15:34       ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Pi-Hsun Shih

Hello Jungo,

I was taking a look at this patch (thanks for the work),
I didn't look in deep details, but I have some comments, please see
below. I hope it helps.

On 12/19/19 3:49 AM, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>  - Correct auto suspend timer value for suspend/resume issue
>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>  - Fix KE due to no sen-inf sub-device
> ---
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++

I think I would split this file a bit, to separate which code is being used for the subdevice, which for
capture, which for metadata, and what is being used to deal with requests.

It would make it easier to review imho.

>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++

It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.

Or maybe something like:

mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
mtkisp_p1_capture.c
mtkisp_p1_meta.c
mtkisp_p1_isp.c
mtkisp_p1_hw.c (or maybe split this between the other files)
mtkisp_p1_request.c
mtkisp_p1_common.c (?)

or s/mtkisp_p1/mtk_cam/

what do you think?

>  9 files changed, 3377 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> new file mode 100644
> index 000000000000..f86e1b59ad1e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/Kconfig
> @@ -0,0 +1,20 @@
> +config VIDEO_MEDIATEK_ISP_PASS1
> +	tristate "Mediatek ISP Pass 1 driver"
> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API

I think you need OF as well

depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF

> +	depends on ARCH_MEDIATEK

depends on ARCH_MEDIATEK || COMPILE_TEST

> +	select V4L2_FWNODE
> +	select VIDEOBUF2_VMALLOC
> +	select VIDEOBUF2_DMA_CONTIG
> +	select MTK_SCP
> +	default n
> +	help
> +		Pass 1 driver controls 3A (auto-focus, exposure,
> +		and white balance) with tuning feature and outputs
> +		the captured image buffers in Mediatek's camera system.
> +
> +		Choose Y if you want to use Mediatek SoCs to create image
> +		captured application such as video recording and still image
> +		capturing.

I would re-word this a bit, since people can use a captured application (and not create one) :)

> +
> +		To compile this driver as a module, choose M here; the module
> +		will be called mtk-cam-isp.
> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> new file mode 100644
> index 000000000000..ce79d283b209
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..53b54d3c26a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +mtk-cam-isp-objs += mtk_cam.o
> +mtk-cam-isp-objs += mtk_cam-hw.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> new file mode 100644
> index 000000000000..4065d0d29b7f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> @@ -0,0 +1,636 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/module.h>
> +#include <linux/remoteproc/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include <media/v4l2-event.h>

Please sort headers alphabetically.

> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +#include "mtk_cam-regs.h"
> +
> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> +#define MTK_ISP_CQ_BUFFER_COUNT			3
> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> +
> +/*
> + *
> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> + * In current driver, only supports CAM B.
> + *
> + */
> +#define MTK_ISP_CAM_ID_B			3
> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> +
> +static void isp_tx_frame_worker(struct work_struct *work)

I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
an external function.

> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> +
> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> +		dev_err(dev, "wrong IPI len:%d\n", len);
> +		return;
> +	}
> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> +		return;
> +
> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> +}
> +
> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI cmd\n");
> +		return ret;
> +	}
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI frame\n");
> +		goto unreg_ipi_cmd;
> +	}
> +
> +	p1_dev->composer_wq =
> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> +					WQ_FREEZABLE);
> +	if (!p1_dev->composer_wq) {
> +		dev_err(dev, "failed to alloc composer workqueue\n");
> +		goto unreg_ipi_frame;
> +	}
> +
> +	return 0;
> +
> +unreg_ipi_frame:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +unreg_ipi_cmd:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +
> +	return ret;
> +}
> +
> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	destroy_workqueue(p1_dev->composer_wq);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +}
> +
> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> +
> +	/*
> +	 * Passed coherent reserved memory info. for SCP firmware usage.
> +	 * This buffer is used for SCP's ISP composer to compose.
> +	 * The size of is fixed to 0x200000 for the requirement of composer.
> +	 */
> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +
> +	isp_composer_uninit(p1_dev);

I think you can copy the 3 lines of this isp_composer_uninit() function here, since
this seems the only place it is being used, and having a deinit and uninit function is
a bit confusing.

> +}
> +
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> +		       struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)

I prefer not having a int parameter, this is easier to read:

mtk_isp_stream_on(cam);
mtk_isp_stream_off(cam);

or

mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);

instead of:

mtk_isp_stream(cam, 1);
mtk_isp_stream(cam, 0);

You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.

> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;

s/is_stream_on/is_streaming

> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		dev_err(dev, "failed to rproc_boot\n");

It would be nice to improve this error message for users, how about:

dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);

Or maybe even remove this message, since rproc_boot() already have several error messages.

> +		return ret;
> +	}
> +
> +	ret = isp_composer_init(p1_dev);
> +	if (ret)

should rproc_shutdown() be called here?

> +		return ret;
> +
> +	pm_runtime_get_sync(dev);

You should check return value here.

> +	isp_composer_hw_init(p1_dev);
> +
> +	p1_dev->enqueued_frame_seq_no = 0;
> +	p1_dev->dequeued_frame_seq_no = 0;
> +	p1_dev->composed_frame_seq_no = 0;
> +	p1_dev->sof_count = 0;
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	isp_composer_hw_deinit(p1_dev);
> +	pm_runtime_mark_last_busy(dev);
> +	pm_runtime_put_autosuspend(dev);
> +	rproc_shutdown(p1_dev->rproc_handle);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}
> +
> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> +			       unsigned int dequeued_frame_seq_no)
> +{
> +	dma_addr_t base_addr = p1_dev->composer_iova;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_cam_dev_request *req;
> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> +	unsigned int addr_offset;
> +
> +	/* Send V4L2_EVENT_FRAME_SYNC event */
> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +
> +	p1_dev->sof_count += 1;
> +	/* Save frame information */
> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> +
> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +	if (req)
> +		req->timestamp = ktime_get_boottime_ns();
> +
> +	/* Update CQ base address if needed */
> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> +		dev_dbg(dev,
> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> +			composed_frame_seq_no, dequeued_frame_seq_no);
> +		return;
> +	}
> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> +}
> +
> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> +{
> +	u32 val;
> +
> +	dev_err(p1_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(p1_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));

I think if would be better to transfor those into dev_dbg and add a counter
in debugfs.

> +
> +	/* Disable DMA error mask to avoid too much error log */
> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> +}
> +
> +static irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> +	struct device *dev = p1_dev->dev;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> +
> +	/*
> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> +	 * If these two ISRs come together, print warning msg to hint.
> +	 */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> +
> +	/* De-queue frame */
> +	if (irq_status & SW_PASS1_DON_ST) {

I suppose this means "done streaming"?

> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> +					      p1_dev->dequeued_frame_seq_no);
> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> +	}
> +
> +	/* Save frame info. & update CQ address for frame HW en-queue */
> +	if (irq_status & SOF_INT_ST)
> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_irq_handle_dma_err(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> +		p1_dev->sof_count, irq_status, dma_status,
> +		dequeued_frame_seq_no);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> +			       struct platform_device *pdev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	dma_addr_t addr;
> +	void *ptr;

Maybe "composer_buffer" would be a better name.

But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?

> +	int ret;
> +
> +	p1_dev->scp = scp_get(pdev);
> +	if (!p1_dev->scp) {
> +		dev_err(dev, "failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);

I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
which mapping we are refering to.

> +
> +	/*
> +	 * Allocate coherent reserved memory for SCP firmware usage.
> +	 * The size of SCP composer's memory is fixed to 0x200000
> +	 * for the requirement of firmware.
> +	 */
> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> +	if (!ptr) {
> +		ret = -ENOMEM;
> +		goto fail_put_scp;
> +	}
> +
> +	p1_dev->composer_scp_addr = addr;
> +	p1_dev->composer_virt_addr = ptr;
> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> +
> +	/*
> +	 * This reserved memory is also be used by ISP P1 HW.
> +	 * Need to get iova address for ISP P1 DMA.
> +	 */
> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map scp iova\n");
> +		ret = -ENOMEM;
> +		goto fail_free_mem;
> +	}
> +	p1_dev->composer_iova = addr;

why not rename this to composer_isp_addr ?
Since, afaik, composer_scp_addr is also iova.

At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).

> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> +
> +	return 0;
> +
> +fail_free_mem:
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +fail_put_scp:
> +	scp_put(p1_dev->scp);
> +
> +	return ret;
> +}
> +
> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> +{
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +	scp_put(p1_dev->scp);
> +}
> +
> +static int mtk_isp_pm_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Disable ISP's view finder and wait for TG idle if possible */
> +	dev_dbg(dev, "cam suspend, disable VF\n");
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> +
> +	/* Disable CMOS */
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Force ISP HW to idle */
> +	ret = pm_runtime_force_suspend(dev);
> +	if (ret) {
> +		dev_err(dev, "failed to force suspend:%d\n", ret);
> +		goto reenable_hw;
> +	}
> +
> +	return 0;
> +
> +reenable_hw:
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_pm_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Force ISP HW to resume */
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret)
> +		return ret;
> +
> +	/* Enable CMOS */
> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Enable VF */
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	dev_dbg(dev, "%s:disable clock\n", __func__);
> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	dev_dbg(dev, "%s:enable clock\n", __func__);
> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to enable clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	/* List of clocks required by isp cam */
> +	static const char * const clk_names[] = {
> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +	};
> +	struct mtk_isp_p1_device *p1_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq, ret, i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	p1_dev->dev = dev;
> +	dev_set_drvdata(dev, p1_dev);
> +
> +	/*
> +	 * Now only support single CAM with CAM B.
> +	 * Get CAM B register base with CAM B index.
> +	 * Support multiple CAMs in future.
> +	 */
> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> +	p1_dev->regs = devm_ioremap_resource(dev, res);
> +	if (IS_ERR(p1_dev->regs)) {
> +		dev_err(dev, "failed to map reister base\n");

s/reister/register

> +		return PTR_ERR(p1_dev->regs);
> +	}
> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> +
> +	/*
> +	 * The cam_sys unit only supports reg., but has no IRQ support.
> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> +	 */
> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> +	if (!irq) {
> +		dev_err(dev, "failed to get irq\n");
> +		return -ENODEV;
> +	}
> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> +			       p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to request irq=%d\n", irq);
> +		return ret;
> +	}
> +	dev_dbg(dev, "registered irq=%d\n", irq);
> +	spin_lock_init(&p1_dev->spinlock_irq);
> +
> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> +	if (!p1_dev->clks)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->num_clks; ++i)
> +		p1_dev->clks[i].id = clk_names[i];
> +
> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> +	pm_runtime_use_autosuspend(dev);
> +	pm_runtime_enable(dev);
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret) {
> +		isp_teardown_scp_rproc(p1_dev);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> +	pm_runtime_dont_use_autosuspend(dev);
> +	pm_runtime_disable(dev);
> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +	isp_teardown_scp_rproc(p1_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> +			   NULL)
> +};
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam-p1",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> new file mode 100644
> index 000000000000..837662f92a5e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h

This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?

> @@ -0,0 +1,64 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_HW_H__
> +#define __MTK_CAM_HW_H__
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ipi.h"
> +
> +/*
> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> + *
> + * @dev: Pointer to device.
> + * @scp_pdev: Pointer to SCP platform device.
> + * @rproc_handle: Pointer to new remoteproc instance.
> + * @cam_dev: Embedded struct cam_dev
> + * @regs: Camera ISP HW base register address
> + * @num_clks: The number of driver's clocks
> + * @clks: The clock data array
> + * @spinlock_irq: Used to protect register read/write data
> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> + * @composed_frame_seq_no: Frame sequence number of composed frame
> + * @timestamp: Frame timestamp in ns
> + * @sof_count: SOF counter
> + * @composer_wq: The work queue for frame request composing
> + * @composer_scp_addr: SCP address of ISP composer memory
> + * @composer_iova: DMA address of ISP composer memory
> + * @virt_addr: Virtual address of ISP composer memory
> + *
> + */
> +struct mtk_isp_p1_device {
> +	struct device *dev;
> +	struct mtk_scp *scp;
> +	struct rproc *rproc_handle;
> +	struct mtk_cam_dev cam_dev;
> +	void __iomem *regs;
> +	unsigned int num_clks;
> +	struct clk_bulk_data *clks;
> +	/* Used to protect register read/write data */
> +	spinlock_t spinlock_irq;
> +	unsigned int enqueued_frame_seq_no;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int composed_frame_seq_no;
> +	u8 sof_count;
> +	struct workqueue_struct *composer_wq;
> +	dma_addr_t composer_scp_addr;
> +	dma_addr_t composer_iova;
> +	void *composer_virt_addr;
> +};
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> +		       struct p1_config_param *config_param);
> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> +			 struct mtk_cam_dev_request *req);

It would be nice to have docs for these too.

> +
> +#endif /* __MTK_CAM_HW_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> new file mode 100644
> index 000000000000..981b634dd91f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h

I'm skipping this file, since, if I understand correctly, this is not ready for review right?

> @@ -0,0 +1,222 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_IPI_H__
> +#define __MTK_CAM_IPI_H__
> +
> +#include <linux/types.h>
> +
> +/*
> + * struct img_size - Image size information.
> + *
> + * @w: Image width, the unit is pixel
> + * @h: Image height, the unit is pixel
> + * @xsize: Bytes per line based on width.
> + * @stride: Bytes per line when changing line.
> + *          Stride is based on xsize + HW constrain(byte align).
> + *
> + */
> +struct img_size {
> +	u32 w;
> +	u32 h;
> +	u32 xsize;
> +	u32 stride;
> +} __packed;
> +
> +/*
> + * struct p1_img_crop - image corp information
> + *
> + * @left: The left of crop area.
> + * @top: The top of crop area.
> + * @width: The width of crop area.
> + * @height: The height of crop area.
> + *
> + */
> +struct p1_img_crop {
> +	u32 left;
> +	u32 top;
> +	u32 width;
> +	u32 height;
> +} __packed;
> +
> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;

I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).

> +	u32 scp_addr;
> +} __packed;
> +
> +/*
> + * struct p1_img_output - ISP P1 image output information
> + *
> + * @buffer: DMA buffer address of image.
> + * @size: The image size configuration.
> + * @crop: The crop configuration.
> + * @pixel_bits: The bits per image pixel.
> + * @img_fmt: The image format.
> + *
> + */
> +struct p1_img_output {
> +	struct dma_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	u8 pixel_bits;
> +	u32 img_fmt;
> +} __packed;
> +
> +/*
> + * struct cfg_in_param - Image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: Indicate the sensor mode. Continuous or single shot.
> + * @subsample: Indicate to enables SOF subsample or not.
> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: Describe input data pattern.
> + * @raw_pixel_id: Bayer sequence.
> + * @tg_fps: The fps rate of TG (time generator).
> + * @img_fmt: The image format of input source.
> + * @p1_img_crop: The crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	u8 continuous;
> +	u8 subsample;
> +	u8 pixel_mode;
> +	u8 data_pattern;
> +	u8 raw_pixel_id;
> +	u16 tg_fps;
> +	u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - The image output parameters of main stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @pure_raw: Indicate the image path control.
> + *            True: pure raw
> + *            False: processing raw
> + * @pure_raw_pack: Indicate the image is packed or not.
> + *                 True: packed mode
> + *                 False: unpacked mode
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	u8 bypass;
> +	u8 pure_raw;
> +	u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - The image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct p1_config_param - ISP P1 configuration parameters.
> + *
> + * @cfg_in_param: The Image input parameters.
> + * @cfg_main_param: The main output image parameters.
> + * @cfg_resize_out_param: The packed output image parameters.
> + * @enabled_dmas: The enabled DMA port information.
> + *
> + */
> +struct p1_config_param {
> +	struct cfg_in_param cfg_in_param;
> +	struct cfg_main_out_param cfg_main_param;
> +	struct cfg_resize_out_param cfg_resize_param;
> +	u32 enabled_dmas;
> +} __packed;
> +
> +/*
> + * struct P1_meta_frame - ISP P1 meta frame information.
> + *
> + * @enabled_dma: The enabled DMA port information.
> + * @vb_index: The VB2 index of meta buffer.
> + * @meta_addr: DMA buffer address of meta buffer.
> + *
> + */
> +struct P1_meta_frame {
> +	u32 enabled_dma;
> +	u32 vb_index;
> +	struct dma_buffer meta_addr;
> +} __packed;
> +
> +/*
> + * struct isp_init_info - ISP P1 composer init information.
> + *
> + * @hw_module: The ISP Camera HW module ID.
> + * @cq_addr: The DMA address of composer memory.
> + *
> + */
> +struct isp_init_info {
> +	u8 hw_module;
> +	struct dma_buffer cq_addr;
> +} __packed;
> +
> +/*
> + * struct isp_ack_info - ISP P1 IPI command ack information.
> + *
> + * @cmd_id: The IPI command ID is acked.
> + * @frame_seq_no: The IPI frame sequence number is acked.
> + *
> + */
> +struct isp_ack_info {
> +	u8 cmd_id;
> +	u32 frame_seq_no;
> +} __packed;
> +
> +/*
> + * The IPI command enumeration.
> + */
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_RESERVED,
> +};
> +
> +/*
> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> + *
> + * @cmd_id: The IPI command ID.
> + * @init_param: The init formation for ISP_CMD_INIT.
> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> + *
> + */
> +struct mtk_isp_scp_p1_cmd {
> +	u8 cmd_id;
> +	union {
> +		struct isp_init_info init_param;
> +		struct p1_config_param config_param;
> +		u32 enabled_dmas;
> +		struct P1_meta_frame meta_frame;
> +		u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +#endif /* __MTK_CAM_IPI_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..ab2277f45fa4
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,95 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_REGS_H__
> +#define __MTK_CAM_REGS_H__
> +
> +/* ISP interrupt enable */
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define DMA_ERR_INT_EN			BIT(29)
> +
> +/* ISP interrupt status */
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define VS_INT_ST			BIT(0)
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define HW_PASS1_DON_ST			BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +#define SW_PASS1_DON_ST			BIT(30)
> +
> +/* ISP interrupt 2 status */
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +
> +/* Configures sensor mode */
> +#define REG_TG_SEN_MODE			0x0230
> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_VF_CON			0x0234
> +#define TG_VF_CON_VFDATA_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_INTER_ST			0x026c
> +#define TG_CS_MASK			0x3f00
> +#define TG_IDLE_ST			BIT(8)
> +
> +/* IMGO error status register */
> +#define REG_IMGO_ERR_STAT		0x1360
> +/* RRZO error status register */
> +#define REG_RRZO_ERR_STAT		0x1364
> +/* AAO error status register */
> +#define REG_AAO_ERR_STAT		0x1368
> +/* AFO error status register */
> +#define REG_AFO_ERR_STAT		0x136c
> +/* LCSO error status register */
> +#define REG_LCSO_ERR_STAT		0x1370
> +/* BPCI error status register */
> +#define REG_BPCI_ERR_STAT		0x137c
> +/* LSCI error status register */
> +#define REG_LSCI_ERR_STAT		0x1384
> +/* LMVO error status register */
> +#define REG_LMVO_ERR_STAT		0x1390
> +/* FLKO error status register */
> +#define REG_FLKO_ERR_STAT		0x1394
> +/* PSO error status register */
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* CQ0 base address */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +/* Frame sequence number */
> +#define REG_FRAME_SEQ_NUM		0x13b8
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +

I would add a common prefix all the registers in the file.

Also, add some docs to know what those acronyms means would be nice.

> +#endif	/* __MTK_CAM_REGS_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..23fdb8b4abc5
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,2087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-dma-contig.h>

Please sort in alphabetical order.

> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_PSO		BIT(10)

It would be nice to have better names of docs of what these means.

> +
> +#define MTK_ISP_ONE_PIXEL_MODE		1
> +#define MTK_ISP_MIN_RESIZE_RATIO	6
> +#define MTK_ISP_MAX_RUNNING_JOBS	3
> +
> +#define MTK_CAM_CIO_PAD_SRC		4
> +#define MTK_CAM_CIO_PAD_SINK		11
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)

no need for the underscore in __vq

> +{
> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_request *
> +mtk_cam_req_to_dev_req(struct media_request *__req)
> +{
> +	return container_of(__req, struct mtk_cam_dev_request, req);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> +				 struct mtk_cam_dev_request *req,
> +				 enum vb2_buffer_state state)
> +{
> +	struct media_request_object *obj, *obj_prev;
> +	unsigned long flags;
> +	u64 ts_eof = ktime_get_boottime_ns();
> +
> +	if (!cam->streaming)

s/streaming/is_streaming

this makes a bit more intuitive of what the the boolean means.

> +		return;
> +
> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> +
> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> +		struct vb2_buffer *vb;
> +		struct mtk_cam_dev_buffer *buf;
> +		struct mtk_cam_video_device *node;
> +
> +		if (!vb2_request_object_is_buffer(obj))
> +			continue;
> +		vb = container_of(obj, struct vb2_buffer, req_obj);
> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +		spin_lock_irqsave(&node->buf_list_lock, flags);
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> +			vb->timestamp = ts_eof;
> +		else
> +			vb->timestamp = req->timestamp;
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +			return req;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +
> +	return NULL;
> +}
> +
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> +				   unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> +			list_del(&req->list);
> +			break;
> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> +				 req->frame_params.frame_seq_no);

maybe a counter in debugfs instead of the warning.

> +			list_del(&req->list);
> +		} else {
> +			break;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	if (!cam->streaming) {
> +		dev_dbg(cam->dev, "stream is off\n");
> +		return;
> +	}
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> +			dev_dbg(cam->dev, "jobs are full\n");
> +			break;
> +		}
> +		cam->running_job_count++;
> +		list_del(&req->list);
> +		list_add_tail(&req->list, &cam->running_job_list);

list_move_tail() can be used.

> +		mtk_isp_req_enqueue(cam, req);
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> +					       media_dev);
> +	unsigned long flags;
> +
> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> +	vb2_request_queue(req);
> +
> +	/* add to pending job list */
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	mtk_cam_dev_req_try_queue(cam);
> +}
> +
> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}

which patchset are these pixel formats defined?
I couldn't find them in the ones you pointed.

I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
so you can use v4l2_format_info() to get the number of bytes.

> +
> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> +			     struct v4l2_pix_format_mplane *mp)
> +{
> +	unsigned int bpl, ppl;

bytes per line and pixels per line right?

> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);

wouldn't be easier a get_pixel_bytes() function instead of bits?

> +	unsigned int width = mp->width;
> +
> +	bpl = 0;
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		/* Bayer encoding format & 2 bytes alignment */
> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		/*
> +		 * The FULL-G encoding format
> +		 * 1 G component per pixel
> +		 * 1 R component per 4 pixel
> +		 * 1 B component per 4 pixel
> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> +		 */
> +		ppl = DIV_ROUND_UP(width * 6, 4);
> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> +
> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> +		if (pixel_bits == 10)
> +			bpl = ALIGN(bpl, 4);
> +		else
> +			bpl = ALIGN(bpl, 8);
> +	}
> +	/*
> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> +	 */
> +	bpl = ALIGN(bpl, 4);
> +
> +	mp->plane_fmt[0].bytesperline = bpl;
> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> +
> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	int i;

unsigned

> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (!queue_desc->image) {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +		return;
> +	}
> +
> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> +	dest->fmt.pix_mp.num_planes = 1;
> +
> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_R;
> +	default:
> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}

I was wondering if it is not better to save all the media bus format
into a table, instead of having several swtch case statements.

> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}> +}

same for the pixelformat.

Then you can cache object with the pixelformat in the main struct.

> +
> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> +			  struct p1_img_output *out_fmt, int sd_width,
> +			  int sd_height)
> +{
> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> +
> +	/* Check output & input image size dimension */
> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> +			node_id);
> +		return -EINVAL;
> +	}
> +
> +	/* Check resize ratio for resize out stream due to HW constraint */
> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> +		return -EINVAL;
> +	}
> +
> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    !out_fmt->pixel_bits) {
> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> +		return -EINVAL;
> +	}
> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> +
> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +	out_fmt->crop.left = 0;
> +	out_fmt->crop.top = 0;
> +	out_fmt->crop.width = sd_width;
> +	out_fmt->crop.height = sd_height;
> +
> +	dev_dbg(cam->dev,
> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		node_id, out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	cam->enabled_count = 0;
> +	cam->enabled_dmas = 0;
> +	cam->stream_count = 0;
> +	cam->running_job_count = 0;
> +
> +	/* Get the enabled meta DMA ports */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		if (!cam->vdev_nodes[i].enabled)
> +			continue;
> +		cam->enabled_count++;
> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> +	}
> +
> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> +		cam->enabled_dmas);
> +}
> +
> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct p1_config_param config_param;
> +	struct cfg_in_param *cfg_in_param;
> +	struct v4l2_subdev_format sd_fmt;
> +	int sd_width, sd_height, sd_code;

are this sd_* variables required? Can't sd_fmt be directly accessed?

> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> +	int ret;
> +
> +	/* Get sensor format configuration */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> +		return ret;
> +	}
> +	sd_width = sd_fmt.format.width;
> +	sd_height = sd_fmt.format.height;
> +	sd_code = sd_fmt.format.code;
> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> +		sd_code);

If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
it should be configured from userspace.

> +
> +	memset(&config_param, 0, sizeof(config_param));
> +
> +	/* Update cfg_in_param */
> +	cfg_in_param = &config_param.cfg_in_param;
> +	cfg_in_param->continuous = true;
> +	/* Fix to one pixel mode in default */
> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> +	cfg_in_param->crop.width = sd_width;
> +	cfg_in_param->crop.height = sd_height;
> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> +		return -EINVAL;
> +	}
> +
> +	/* Update cfg_main_param */
> +	config_param.cfg_main_param.pure_raw = true;
> +	config_param.cfg_main_param.pure_raw_pack = true;
> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> +			     &config_param.cfg_main_param.output,
> +			     sd_width, sd_height);
> +	if (ret)
> +		return ret;
> +
> +	/* Update cfg_resize_param */
> +	if (enabled_dma_ports & R_RRZO) {
> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> +				     &config_param.cfg_resize_param.output,
> +				     sd_width, sd_height);
> +		if (ret)
> +			return ret;
> +	} else {
> +		config_param.cfg_resize_param.bypass = true;
> +	}
> +
> +	/* Update enabled_dmas */
> +	config_param.enabled_dmas = enabled_dma_ports;
> +	mtk_isp_hw_config(cam, &config_param);
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> +				  unsigned int frame_seq_no)
> +{
> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = frame_seq_no,
> +	};
> +
> +	v4l2_event_queue(cam->subdev.devnode, &event);
> +}
> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> +{
> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> +	struct device *dev = cam->dev;
> +	struct media_entity *entity;
> +	struct v4l2_subdev *sensor;
> +
> +	sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> +			entity->name, entity->function, entity->stream_count);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "no seninf connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "no seninf connected\n");
> +		return -ENODEV;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> +	if (!cam->sensor)
> +		return -ENODEV;
> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		goto fail_seninf_off;
> +	}
> +
> +	ret = mtk_cam_dev_isp_config(cam);
> +	if (ret)
> +		goto fail_sensor_off;
> +
> +	cam->streaming = true;
> +	mtk_isp_stream(cam, 1);
> +	mtk_cam_dev_req_try_queue(cam);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +fail_seninf_off:
> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		return -EPERM;

Why -EPERM ?

> +	}
> +
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	cam->streaming = false;
> +	mtk_isp_stream(cam, 0);
> +	mtk_isp_hw_release(cam);
> +
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> +
> +	if (enable) {
> +		/* Align vb2_core_streamon design */
> +		if (cam->streaming) {
> +			dev_warn(cam->dev, "already streaming on\n");

I think just dev_dbg is enough.

> +			return 0;
> +		}
> +		return mtk_cam_cio_stream_on(cam);
> +	}
> +
> +	if (!cam->streaming) {
> +		dev_warn(cam->dev, "already streaming off\n");

same here

> +		return 0;
> +	}
> +	return mtk_cam_cio_stream_off(cam);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	/*
> +	 * The video nodes exposed by the driver have pads indexes
> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> +	 */
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> +		cam->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);

Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?

> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;

isn't it an issue if userspace queue two buffers for the same video device in the same request?

vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
will be at req->frame_params.dma_bufs[buf->node_id], no?

Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
buffer?

Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.

> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	dma_addr_t addr;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> +	buf->scp_addr = 0;
> +
> +	/* SCP address is only valid for meta input buffer */
> +	if (!node->desc.smem_alloc)
> +		return 0;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	/* Use coherent address to get iova address */
> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> +		return -EFAULT;
> +	}
> +	buf->scp_addr = buf->daddr;
> +	buf->daddr = addr;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> +		if (vb2_get_plane_payload(vb, 0) != size) {
> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> +				vb2_get_plane_payload(vb, 0), size);
> +			return -EINVAL;
> +		}
> +		return 0;
> +	}
> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +	struct device *dev = cam->dev;
> +
> +	if (!node->desc.smem_alloc)
> +		return;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	dma_unmap_page_attrs(dev, buf->daddr,
> +			     vb->planes[0].length,
> +			     DMA_BIDIRECTIONAL,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +}
> +
> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (node->desc.smem_alloc)
> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {
> +		if (sizes[0] < size || *num_planes != 1)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_ret_buf;
> +	}
> +
> +	mutex_lock(&cam->op_lock);
> +	/* Start streaming of the whole pipeline now*/
> +	if (!cam->pipeline.streaming_count) {

No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.

> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> +		if (ret) {
> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> +			goto fail_unlock;
> +		}
> +		mtk_cam_dev_init_stream(cam);
> +		ret = mtk_isp_hw_init(cam);
> +		if (ret) {
> +			dev_err(dev, "failed to init HW:%d\n", ret);
> +			goto fail_stop_pipeline;
> +		}
> +	}
> +
> +	/* Media links are fixed after media_pipeline_start */
> +	cam->stream_count++;
> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> +		cam->enabled_count);
> +	if (cam->stream_count < cam->enabled_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return 0;
> +	}
> +
> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> +	if (ret)
> +		goto fail_no_stream;
> +	mutex_unlock(&cam->op_lock);
> +
> +	return 0;
> +
> +fail_no_stream:
> +	cam->stream_count--;
> +fail_stop_pipeline:
> +	if (cam->stream_count == 0)
> +		media_pipeline_stop(&node->vdev.entity);
> +fail_unlock:
> +	mutex_unlock(&cam->op_lock);
> +fail_ret_buf:
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +
> +	mutex_lock(&cam->op_lock);
> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> +		cam->stream_count);
> +	/* Check the first node to stream-off */
> +	if (cam->stream_count == cam->enabled_count)
> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> +
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> +	cam->stream_count--;
> +	if (cam->stream_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return;
> +	}
> +	mutex_unlock(&cam->op_lock);
> +
> +	mtk_cam_dev_req_cleanup(cam);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +
> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam->dev));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct device *dev = cam->dev;
> +	const struct v4l2_format *dev_fmt;
> +	struct v4l2_format try_fmt;
> +
> +	memset(&try_fmt, 0, sizeof(try_fmt));
> +	try_fmt.type = f->type;
> +
> +	/* Validate pixelformat */
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> +	if (!dev_fmt) {
> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> +	}
> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> +
> +	/* Validate image width & height range */
> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> +	/* 4 bytes alignment for width */
> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> +
> +	/* Only support one plane */
> +	try_fmt.fmt.pix_mp.num_planes = 1;
> +
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> +
> +	/* Constant format fields */
> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +
> +	*f = try_fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (vb2_is_busy(node->vdev.queue)) {
> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> +		return -EBUSY;
> +	}
> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +	/* Configure to video device */
> +	node->vdev_fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};

hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
I wonder if pad ops shouldn't be implemented too (to be verified).

> +
> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> +	.buf_request_complete = mtk_cam_vb2_request_complete,
> +};> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_ops = {
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> +				  struct media_device *media_dev)
> +{
> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	media_dev->dev = cam->dev;
> +	strscpy(media_dev->model, dev_driver_string(dev),
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_ops;
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize subdev pads */
> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> +					sizeof(*cam->subdev_pads),
> +					GFP_KERNEL);
> +	if (!cam->subdev_pads) {
> +		dev_err(dev, "failed to allocate subdev_pads\n");
> +		ret = -ENOMEM;
> +		goto fail_media_unreg;
> +	}
> +
> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> +				     cam->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_pads; i++)
> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	return 0;
> +
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int
> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> +			      struct mtk_cam_video_device *node)
> +{
> +	struct device *dev = cam->dev;
> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> +	unsigned int link_flags = node->desc.link_flags;
> +	int ret;
> +
> +	/* Initialize mtk_cam_video_device */
> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> +		node->enabled = true;
> +	else
> +		node->enabled = false;
> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> +
> +	cam->subdev_pads[node->id].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize vbq */
> +	vbq->type = node->desc.buf_type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam->smem_dev;
> +	} else {
> +		vbq->dev = dev;
> +	}
> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> +	if (output)
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> +	else
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> +	/* No minimum buffers limitation */
> +	vbq->min_buffers_needed = 0;
> +	vbq->drv_priv = cam;
> +	vbq->lock = &node->vdev_lock;
> +	vbq->supports_requests = true;
> +	vbq->requires_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_media_clean;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 dev_driver_string(dev), node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->vdev_lock;
> +	vdev->v4l2_dev = &cam->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> +	vdev->entity.ops = NULL;
> +	video_set_drvdata(vdev, cam);
> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->vdev_lock);
> +	INIT_LIST_HEAD(&node->buf_list);
> +	spin_lock_init(&node->buf_list_lock);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vb2_rel;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam->subdev.entity,
> +					    node->id, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam->subdev.entity,
> +					    node->id, &vdev->entity, 0,
> +					    link_flags);
> +	}

No need for the curly braces.

> +	if (ret)
> +		goto fail_vdev_ureg;
> +
> +	return 0;
> +
> +fail_vdev_ureg:
> +	video_unregister_device(vdev);
> +fail_vb2_rel:
> +	mutex_destroy(&node->vdev_lock);
> +	vb2_queue_release(vbq);
> +fail_media_clean:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static void
> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> +{
> +	video_unregister_device(&node->vdev);
> +	vb2_queue_release(&node->vbq);
> +	media_entity_cleanup(&node->vdev.entity);
> +	mutex_destroy(&node->vdev_lock);
> +}
> +
> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	/* Set up media device & pads */
> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> +	if (ret)
> +		return ret;
> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> +
> +	/* Set up v4l2 device */
> +	cam->v4l2_dev.mdev = &cam->media_dev;
> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> +		 "%s", dev_driver_string(dev));
> +	v4l2_set_subdevdata(&cam->subdev, cam);
> +
> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> +		goto fail_clean_media_entiy;
> +	}
> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> +
> +		node->id = node->desc.id;
> +		ret = mtk_cam_video_register_device(cam, node);
> +		if (ret)
> +			goto fail_vdev_unreg;
> +	}
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_vdev_unreg:
> +	for (i--; i >= 0; i--)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +fail_clean_media_entiy:
> +	media_entity_cleanup(&cam->subdev.entity);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +
> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> +	v4l2_device_unregister_subdev(&cam->subdev);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +	media_entity_cleanup(&cam->subdev.entity);
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> +		return -ENODEV;
> +	}
> +
> +	cam->seninf = sd;
> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam->seninf = NULL;
> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "No seninf subdev\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> +				    MEDIA_LNK_FL_IMMUTABLE |
> +				    MEDIA_LNK_FL_ENABLED);
> +	if (ret) {
> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> +			cam->seninf->entity.name, cam->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	v4l2_async_notifier_init(&cam->notifier);
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);

It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/

Regards,
Helen

> +	if (ret) {
> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> +		return ret;
> +	}
> +
> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> +	if (ret) {
> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> +{
> +	v4l2_async_notifier_unregister(&cam->notifier);
> +	v4l2_async_notifier_cleanup(&cam->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 512 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 1200 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 640 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 288 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	/* This is a default image format */
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> +{
> +	unsigned int node_idx;
> +	int i;
> +
> +	node_idx = 0;
> +	/* Setup the output queue */
> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam)
> +{
> +	int ret;
> +
> +	cam->dev = &pdev->dev;
> +	mtk_cam_dev_queue_setup(cam);
> +
> +	spin_lock_init(&cam->pending_job_lock);
> +	spin_lock_init(&cam->running_job_lock);
> +	INIT_LIST_HEAD(&cam->pending_job_list);
> +	INIT_LIST_HEAD(&cam->running_job_list);
> +	mutex_init(&cam->op_lock);
> +
> +	/* v4l2 sub-device registration */
> +	ret = mtk_cam_v4l2_register(cam);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam);
> +	if (ret)
> +		goto fail_v4l2_unreg;
> +
> +	return 0;
> +
> +fail_v4l2_unreg:
> +	mutex_destroy(&cam->op_lock);
> +	mtk_cam_v4l2_unregister(cam);
> +
> +	return ret;
> +}
> +
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> +{
> +	mtk_cam_v4l2_async_unregister(cam);
> +	mtk_cam_v4l2_unregister(cam);
> +	mutex_destroy(&cam->op_lock);
> +}
> +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..0a340a1e65ea
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,244 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_H__
> +#define __MTK_CAM_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "mtk_cam-ipi.h"
> +
> +#define IMG_MAX_WIDTH		5376
> +#define IMG_MAX_HEIGHT		4032
> +#define IMG_MIN_WIDTH		80
> +#define IMG_MIN_HEIGHT		60
> +
> +/*
> + * ID enum value for struct mtk_cam_dev_node_desc:id
> + * or mtk_cam_video_device:id
> + */
> +enum  {
> +	MTK_CAM_P1_META_IN_0 = 0,
> +	MTK_CAM_P1_MAIN_STREAM_OUT,
> +	MTK_CAM_P1_PACKED_BIN_OUT,
> +	MTK_CAM_P1_META_OUT_0,
> +	MTK_CAM_P1_META_OUT_1,
> +	MTK_CAM_P1_META_OUT_2,
> +	MTK_CAM_P1_META_OUT_3,
> +	MTK_CAM_P1_TOTAL_NODES
> +};
> +
> +/* Supported image format list */
> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> +
> +/* Supported bayer pixel order */
> +#define MTK_CAM_RAW_PXL_ID_B		0
> +#define MTK_CAM_RAW_PXL_ID_GB		1
> +#define MTK_CAM_RAW_PXL_ID_GR		2
> +#define MTK_CAM_RAW_PXL_ID_R		3
> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> +
> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +/*
> + * struct mtk_cam_dev_request - MTK camera device request.
> + *
> + * @req: Embedded struct media request.
> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> + * @frame_work: work queue entry for frame transmission to SCP.
> + * @list: List entry of the object for @struct mtk_cam_dev:
> + *        pending_job_list or running_job_list.
> + * @timestamp: Start of frame timestamp in ns
> + *
> + */
> +struct mtk_cam_dev_request {
> +	struct media_request req;
> +	struct mtk_p1_frame_param frame_params;
> +	struct work_struct frame_work;
> +	struct list_head list;
> +	u64 timestamp;
> +};
> +
> +/*
> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> + *
> + * @vbb: Embedded struct vb2_v4l2_buffer.
> + * @list: List entry of the object for @struct mtk_cam_video_device:
> + *        buf_list.
> + * @daddr: The DMA address of this buffer.
> + * @scp_addr: The SCP address of this buffer which
> + *            is only supported for meta input node.
> + * @node_id: The vidoe node id which this buffer belongs to.
> + *
> + */
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer vbb;
> +	struct list_head list;
> +	/* Intenal part */
> +	dma_addr_t daddr;
> +	dma_addr_t scp_addr;
> +	unsigned int node_id;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> + *
> + * @id: id of the node
> + * @name: name of the node
> + * @cap: supported V4L2 capabilities
> + * @buf_type: supported V4L2 buffer type
> + * @dma_port: the dma ports associated to the node
> + * @link_flags: default media link flags
> + * @smem_alloc: using the smem_dev as alloc device or not
> + * @image: true for image node, false for meta node
> + * @num_fmts: the number of supported node formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum VB2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts: supported format
> + * @frmsizes: supported V4L2 frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	const char *name;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure
> + *
> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> + * @enabled: Indicate the video device is enabled or not
> + * @desc: The node description of video device
> + * @vdev_fmt: The V4L2 format of video device
> + * @vdev_pad: The media pad graph object of video device
> + * @vbq: A videobuf queue of video device
> + * @vdev: The video device instance
> + * @vdev_lock: Serializes vb2 queue and video device operations
> + * @buf_list: List for enqueue buffers
> + * @buf_list_lock: Lock used to protect buffer list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_format vdev_fmt;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct video_device vdev;
> +	/* Serializes vb2 queue and video device operations */
> +	struct mutex vdev_lock;
> +	struct list_head buf_list;
> +	/* Lock used to protect buffer list */
> +	spinlock_t buf_list_lock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @dev: Pointer to device.
> + * @smem_pdev: Pointer to shared memory device.
> + * @pipeline: Media pipeline information.
> + * @media_dev: Media device instance.
> + * @subdev: The V4L2 sub-device instance.
> + * @v4l2_dev: The V4L2 device driver instance.
> + * @notifier: The v4l2_device notifier data.
> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> + * @seninf: Pointer to the seninf sub-device.
> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> + * @streaming: Indicate the overall streaming status is on or off.
> + * @enabled_dmas: The enabled dma port information when streaming on.
> + * @enabled_count: Number of enabled video nodes
> + * @stream_count: Number of streaming video nodes
> + * @running_job_count: Nunber of running jobs in the HW driver.
> + * @pending_job_list: List to keep the media requests before en-queue into
> + *                    HW driver.
> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> + * @running_job_list: List to keep the media requests after en-queue into
> + *                    HW driver.
> + * @running_job_lock: Protect the running_job_list data.
> + * @op_lock: Serializes driver's VB2 callback operations.
> + *
> + */
> +struct mtk_cam_dev {
> +	struct device *dev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int enabled_dmas;
> +	unsigned int enabled_count;
> +	unsigned int stream_count;
> +	unsigned int running_job_count;
> +	struct list_head pending_job_list;
> +	/* Protect the pending_job_list data */
> +	spinlock_t pending_job_lock;
> +	struct list_head running_job_list;
> +	/* Protect the running_job_list data & running_job_count */
> +	spinlock_t running_job_lock;
> +	/* Serializes driver's VB2 callback operations */
> +	struct mutex op_lock;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> +				   unsigned int frame_seq_no);
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				  unsigned int frame_seq_no);
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no);
> +
> +#endif /* __MTK_CAM_H__ */
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-03-31 15:34       ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-03-31 15:34 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hello Jungo,

I was taking a look at this patch (thanks for the work),
I didn't look in deep details, but I have some comments, please see
below. I hope it helps.

On 12/19/19 3:49 AM, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>  - Correct auto suspend timer value for suspend/resume issue
>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>  - Fix KE due to no sen-inf sub-device
> ---
>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++

I think I would split this file a bit, to separate which code is being used for the subdevice, which for
capture, which for metadata, and what is being used to deal with requests.

It would make it easier to review imho.

>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++

It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.

Or maybe something like:

mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
mtkisp_p1_capture.c
mtkisp_p1_meta.c
mtkisp_p1_isp.c
mtkisp_p1_hw.c (or maybe split this between the other files)
mtkisp_p1_request.c
mtkisp_p1_common.c (?)

or s/mtkisp_p1/mtk_cam/

what do you think?

>  9 files changed, 3377 insertions(+)
>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> new file mode 100644
> index 000000000000..f86e1b59ad1e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/Kconfig
> @@ -0,0 +1,20 @@
> +config VIDEO_MEDIATEK_ISP_PASS1
> +	tristate "Mediatek ISP Pass 1 driver"
> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API

I think you need OF as well

depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF

> +	depends on ARCH_MEDIATEK

depends on ARCH_MEDIATEK || COMPILE_TEST

> +	select V4L2_FWNODE
> +	select VIDEOBUF2_VMALLOC
> +	select VIDEOBUF2_DMA_CONTIG
> +	select MTK_SCP
> +	default n
> +	help
> +		Pass 1 driver controls 3A (auto-focus, exposure,
> +		and white balance) with tuning feature and outputs
> +		the captured image buffers in Mediatek's camera system.
> +
> +		Choose Y if you want to use Mediatek SoCs to create image
> +		captured application such as video recording and still image
> +		capturing.

I would re-word this a bit, since people can use a captured application (and not create one) :)

> +
> +		To compile this driver as a module, choose M here; the module
> +		will be called mtk-cam-isp.
> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> new file mode 100644
> index 000000000000..ce79d283b209
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..53b54d3c26a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +mtk-cam-isp-objs += mtk_cam.o
> +mtk-cam-isp-objs += mtk_cam-hw.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> new file mode 100644
> index 000000000000..4065d0d29b7f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> @@ -0,0 +1,636 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/module.h>
> +#include <linux/remoteproc/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include <media/v4l2-event.h>

Please sort headers alphabetically.

> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +#include "mtk_cam-regs.h"
> +
> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> +#define MTK_ISP_CQ_BUFFER_COUNT			3
> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> +
> +/*
> + *
> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> + * In current driver, only supports CAM B.
> + *
> + */
> +#define MTK_ISP_CAM_ID_B			3
> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> +
> +static void isp_tx_frame_worker(struct work_struct *work)

I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
an external function.

> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> +
> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> +		dev_err(dev, "wrong IPI len:%d\n", len);
> +		return;
> +	}
> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> +		return;
> +
> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> +}
> +
> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI cmd\n");
> +		return ret;
> +	}
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI frame\n");
> +		goto unreg_ipi_cmd;
> +	}
> +
> +	p1_dev->composer_wq =
> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> +					WQ_FREEZABLE);
> +	if (!p1_dev->composer_wq) {
> +		dev_err(dev, "failed to alloc composer workqueue\n");
> +		goto unreg_ipi_frame;
> +	}
> +
> +	return 0;
> +
> +unreg_ipi_frame:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +unreg_ipi_cmd:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +
> +	return ret;
> +}
> +
> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	destroy_workqueue(p1_dev->composer_wq);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +}
> +
> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> +
> +	/*
> +	 * Passed coherent reserved memory info. for SCP firmware usage.
> +	 * This buffer is used for SCP's ISP composer to compose.
> +	 * The size of is fixed to 0x200000 for the requirement of composer.
> +	 */
> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +
> +	isp_composer_uninit(p1_dev);

I think you can copy the 3 lines of this isp_composer_uninit() function here, since
this seems the only place it is being used, and having a deinit and uninit function is
a bit confusing.

> +}
> +
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> +		       struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)

I prefer not having a int parameter, this is easier to read:

mtk_isp_stream_on(cam);
mtk_isp_stream_off(cam);

or

mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);

instead of:

mtk_isp_stream(cam, 1);
mtk_isp_stream(cam, 0);

You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.

> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;

s/is_stream_on/is_streaming

> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		dev_err(dev, "failed to rproc_boot\n");

It would be nice to improve this error message for users, how about:

dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);

Or maybe even remove this message, since rproc_boot() already have several error messages.

> +		return ret;
> +	}
> +
> +	ret = isp_composer_init(p1_dev);
> +	if (ret)

should rproc_shutdown() be called here?

> +		return ret;
> +
> +	pm_runtime_get_sync(dev);

You should check return value here.

> +	isp_composer_hw_init(p1_dev);
> +
> +	p1_dev->enqueued_frame_seq_no = 0;
> +	p1_dev->dequeued_frame_seq_no = 0;
> +	p1_dev->composed_frame_seq_no = 0;
> +	p1_dev->sof_count = 0;
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	isp_composer_hw_deinit(p1_dev);
> +	pm_runtime_mark_last_busy(dev);
> +	pm_runtime_put_autosuspend(dev);
> +	rproc_shutdown(p1_dev->rproc_handle);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}
> +
> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> +			       unsigned int dequeued_frame_seq_no)
> +{
> +	dma_addr_t base_addr = p1_dev->composer_iova;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_cam_dev_request *req;
> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> +	unsigned int addr_offset;
> +
> +	/* Send V4L2_EVENT_FRAME_SYNC event */
> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +
> +	p1_dev->sof_count += 1;
> +	/* Save frame information */
> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> +
> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +	if (req)
> +		req->timestamp = ktime_get_boottime_ns();
> +
> +	/* Update CQ base address if needed */
> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> +		dev_dbg(dev,
> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> +			composed_frame_seq_no, dequeued_frame_seq_no);
> +		return;
> +	}
> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> +}
> +
> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> +{
> +	u32 val;
> +
> +	dev_err(p1_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(p1_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));

I think if would be better to transfor those into dev_dbg and add a counter
in debugfs.

> +
> +	/* Disable DMA error mask to avoid too much error log */
> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> +}
> +
> +static irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> +	struct device *dev = p1_dev->dev;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> +
> +	/*
> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> +	 * If these two ISRs come together, print warning msg to hint.
> +	 */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> +
> +	/* De-queue frame */
> +	if (irq_status & SW_PASS1_DON_ST) {

I suppose this means "done streaming"?

> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> +					      p1_dev->dequeued_frame_seq_no);
> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> +	}
> +
> +	/* Save frame info. & update CQ address for frame HW en-queue */
> +	if (irq_status & SOF_INT_ST)
> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_irq_handle_dma_err(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> +		p1_dev->sof_count, irq_status, dma_status,
> +		dequeued_frame_seq_no);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> +			       struct platform_device *pdev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	dma_addr_t addr;
> +	void *ptr;

Maybe "composer_buffer" would be a better name.

But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?

> +	int ret;
> +
> +	p1_dev->scp = scp_get(pdev);
> +	if (!p1_dev->scp) {
> +		dev_err(dev, "failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);

I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
which mapping we are refering to.

> +
> +	/*
> +	 * Allocate coherent reserved memory for SCP firmware usage.
> +	 * The size of SCP composer's memory is fixed to 0x200000
> +	 * for the requirement of firmware.
> +	 */
> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> +	if (!ptr) {
> +		ret = -ENOMEM;
> +		goto fail_put_scp;
> +	}
> +
> +	p1_dev->composer_scp_addr = addr;
> +	p1_dev->composer_virt_addr = ptr;
> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> +
> +	/*
> +	 * This reserved memory is also be used by ISP P1 HW.
> +	 * Need to get iova address for ISP P1 DMA.
> +	 */
> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map scp iova\n");
> +		ret = -ENOMEM;
> +		goto fail_free_mem;
> +	}
> +	p1_dev->composer_iova = addr;

why not rename this to composer_isp_addr ?
Since, afaik, composer_scp_addr is also iova.

At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).

> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> +
> +	return 0;
> +
> +fail_free_mem:
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +fail_put_scp:
> +	scp_put(p1_dev->scp);
> +
> +	return ret;
> +}
> +
> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> +{
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +	scp_put(p1_dev->scp);
> +}
> +
> +static int mtk_isp_pm_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Disable ISP's view finder and wait for TG idle if possible */
> +	dev_dbg(dev, "cam suspend, disable VF\n");
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> +
> +	/* Disable CMOS */
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Force ISP HW to idle */
> +	ret = pm_runtime_force_suspend(dev);
> +	if (ret) {
> +		dev_err(dev, "failed to force suspend:%d\n", ret);
> +		goto reenable_hw;
> +	}
> +
> +	return 0;
> +
> +reenable_hw:
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_pm_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Force ISP HW to resume */
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret)
> +		return ret;
> +
> +	/* Enable CMOS */
> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Enable VF */
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	dev_dbg(dev, "%s:disable clock\n", __func__);
> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	dev_dbg(dev, "%s:enable clock\n", __func__);
> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to enable clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	/* List of clocks required by isp cam */
> +	static const char * const clk_names[] = {
> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +	};
> +	struct mtk_isp_p1_device *p1_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq, ret, i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	p1_dev->dev = dev;
> +	dev_set_drvdata(dev, p1_dev);
> +
> +	/*
> +	 * Now only support single CAM with CAM B.
> +	 * Get CAM B register base with CAM B index.
> +	 * Support multiple CAMs in future.
> +	 */
> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> +	p1_dev->regs = devm_ioremap_resource(dev, res);
> +	if (IS_ERR(p1_dev->regs)) {
> +		dev_err(dev, "failed to map reister base\n");

s/reister/register

> +		return PTR_ERR(p1_dev->regs);
> +	}
> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> +
> +	/*
> +	 * The cam_sys unit only supports reg., but has no IRQ support.
> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> +	 */
> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> +	if (!irq) {
> +		dev_err(dev, "failed to get irq\n");
> +		return -ENODEV;
> +	}
> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> +			       p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to request irq=%d\n", irq);
> +		return ret;
> +	}
> +	dev_dbg(dev, "registered irq=%d\n", irq);
> +	spin_lock_init(&p1_dev->spinlock_irq);
> +
> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> +	if (!p1_dev->clks)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->num_clks; ++i)
> +		p1_dev->clks[i].id = clk_names[i];
> +
> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> +	pm_runtime_use_autosuspend(dev);
> +	pm_runtime_enable(dev);
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret) {
> +		isp_teardown_scp_rproc(p1_dev);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> +	pm_runtime_dont_use_autosuspend(dev);
> +	pm_runtime_disable(dev);
> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +	isp_teardown_scp_rproc(p1_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> +			   NULL)
> +};
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam-p1",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> new file mode 100644
> index 000000000000..837662f92a5e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h

This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?

> @@ -0,0 +1,64 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_HW_H__
> +#define __MTK_CAM_HW_H__
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ipi.h"
> +
> +/*
> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> + *
> + * @dev: Pointer to device.
> + * @scp_pdev: Pointer to SCP platform device.
> + * @rproc_handle: Pointer to new remoteproc instance.
> + * @cam_dev: Embedded struct cam_dev
> + * @regs: Camera ISP HW base register address
> + * @num_clks: The number of driver's clocks
> + * @clks: The clock data array
> + * @spinlock_irq: Used to protect register read/write data
> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> + * @composed_frame_seq_no: Frame sequence number of composed frame
> + * @timestamp: Frame timestamp in ns
> + * @sof_count: SOF counter
> + * @composer_wq: The work queue for frame request composing
> + * @composer_scp_addr: SCP address of ISP composer memory
> + * @composer_iova: DMA address of ISP composer memory
> + * @virt_addr: Virtual address of ISP composer memory
> + *
> + */
> +struct mtk_isp_p1_device {
> +	struct device *dev;
> +	struct mtk_scp *scp;
> +	struct rproc *rproc_handle;
> +	struct mtk_cam_dev cam_dev;
> +	void __iomem *regs;
> +	unsigned int num_clks;
> +	struct clk_bulk_data *clks;
> +	/* Used to protect register read/write data */
> +	spinlock_t spinlock_irq;
> +	unsigned int enqueued_frame_seq_no;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int composed_frame_seq_no;
> +	u8 sof_count;
> +	struct workqueue_struct *composer_wq;
> +	dma_addr_t composer_scp_addr;
> +	dma_addr_t composer_iova;
> +	void *composer_virt_addr;
> +};
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> +		       struct p1_config_param *config_param);
> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> +			 struct mtk_cam_dev_request *req);

It would be nice to have docs for these too.

> +
> +#endif /* __MTK_CAM_HW_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> new file mode 100644
> index 000000000000..981b634dd91f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h

I'm skipping this file, since, if I understand correctly, this is not ready for review right?

> @@ -0,0 +1,222 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_IPI_H__
> +#define __MTK_CAM_IPI_H__
> +
> +#include <linux/types.h>
> +
> +/*
> + * struct img_size - Image size information.
> + *
> + * @w: Image width, the unit is pixel
> + * @h: Image height, the unit is pixel
> + * @xsize: Bytes per line based on width.
> + * @stride: Bytes per line when changing line.
> + *          Stride is based on xsize + HW constrain(byte align).
> + *
> + */
> +struct img_size {
> +	u32 w;
> +	u32 h;
> +	u32 xsize;
> +	u32 stride;
> +} __packed;
> +
> +/*
> + * struct p1_img_crop - image corp information
> + *
> + * @left: The left of crop area.
> + * @top: The top of crop area.
> + * @width: The width of crop area.
> + * @height: The height of crop area.
> + *
> + */
> +struct p1_img_crop {
> +	u32 left;
> +	u32 top;
> +	u32 width;
> +	u32 height;
> +} __packed;
> +
> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;

I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).

> +	u32 scp_addr;
> +} __packed;
> +
> +/*
> + * struct p1_img_output - ISP P1 image output information
> + *
> + * @buffer: DMA buffer address of image.
> + * @size: The image size configuration.
> + * @crop: The crop configuration.
> + * @pixel_bits: The bits per image pixel.
> + * @img_fmt: The image format.
> + *
> + */
> +struct p1_img_output {
> +	struct dma_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	u8 pixel_bits;
> +	u32 img_fmt;
> +} __packed;
> +
> +/*
> + * struct cfg_in_param - Image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: Indicate the sensor mode. Continuous or single shot.
> + * @subsample: Indicate to enables SOF subsample or not.
> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: Describe input data pattern.
> + * @raw_pixel_id: Bayer sequence.
> + * @tg_fps: The fps rate of TG (time generator).
> + * @img_fmt: The image format of input source.
> + * @p1_img_crop: The crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	u8 continuous;
> +	u8 subsample;
> +	u8 pixel_mode;
> +	u8 data_pattern;
> +	u8 raw_pixel_id;
> +	u16 tg_fps;
> +	u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - The image output parameters of main stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @pure_raw: Indicate the image path control.
> + *            True: pure raw
> + *            False: processing raw
> + * @pure_raw_pack: Indicate the image is packed or not.
> + *                 True: packed mode
> + *                 False: unpacked mode
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	u8 bypass;
> +	u8 pure_raw;
> +	u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - The image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct p1_config_param - ISP P1 configuration parameters.
> + *
> + * @cfg_in_param: The Image input parameters.
> + * @cfg_main_param: The main output image parameters.
> + * @cfg_resize_out_param: The packed output image parameters.
> + * @enabled_dmas: The enabled DMA port information.
> + *
> + */
> +struct p1_config_param {
> +	struct cfg_in_param cfg_in_param;
> +	struct cfg_main_out_param cfg_main_param;
> +	struct cfg_resize_out_param cfg_resize_param;
> +	u32 enabled_dmas;
> +} __packed;
> +
> +/*
> + * struct P1_meta_frame - ISP P1 meta frame information.
> + *
> + * @enabled_dma: The enabled DMA port information.
> + * @vb_index: The VB2 index of meta buffer.
> + * @meta_addr: DMA buffer address of meta buffer.
> + *
> + */
> +struct P1_meta_frame {
> +	u32 enabled_dma;
> +	u32 vb_index;
> +	struct dma_buffer meta_addr;
> +} __packed;
> +
> +/*
> + * struct isp_init_info - ISP P1 composer init information.
> + *
> + * @hw_module: The ISP Camera HW module ID.
> + * @cq_addr: The DMA address of composer memory.
> + *
> + */
> +struct isp_init_info {
> +	u8 hw_module;
> +	struct dma_buffer cq_addr;
> +} __packed;
> +
> +/*
> + * struct isp_ack_info - ISP P1 IPI command ack information.
> + *
> + * @cmd_id: The IPI command ID is acked.
> + * @frame_seq_no: The IPI frame sequence number is acked.
> + *
> + */
> +struct isp_ack_info {
> +	u8 cmd_id;
> +	u32 frame_seq_no;
> +} __packed;
> +
> +/*
> + * The IPI command enumeration.
> + */
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_RESERVED,
> +};
> +
> +/*
> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> + *
> + * @cmd_id: The IPI command ID.
> + * @init_param: The init formation for ISP_CMD_INIT.
> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> + *
> + */
> +struct mtk_isp_scp_p1_cmd {
> +	u8 cmd_id;
> +	union {
> +		struct isp_init_info init_param;
> +		struct p1_config_param config_param;
> +		u32 enabled_dmas;
> +		struct P1_meta_frame meta_frame;
> +		u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +#endif /* __MTK_CAM_IPI_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..ab2277f45fa4
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,95 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_REGS_H__
> +#define __MTK_CAM_REGS_H__
> +
> +/* ISP interrupt enable */
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define DMA_ERR_INT_EN			BIT(29)
> +
> +/* ISP interrupt status */
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define VS_INT_ST			BIT(0)
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define HW_PASS1_DON_ST			BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +#define SW_PASS1_DON_ST			BIT(30)
> +
> +/* ISP interrupt 2 status */
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +
> +/* Configures sensor mode */
> +#define REG_TG_SEN_MODE			0x0230
> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_VF_CON			0x0234
> +#define TG_VF_CON_VFDATA_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_INTER_ST			0x026c
> +#define TG_CS_MASK			0x3f00
> +#define TG_IDLE_ST			BIT(8)
> +
> +/* IMGO error status register */
> +#define REG_IMGO_ERR_STAT		0x1360
> +/* RRZO error status register */
> +#define REG_RRZO_ERR_STAT		0x1364
> +/* AAO error status register */
> +#define REG_AAO_ERR_STAT		0x1368
> +/* AFO error status register */
> +#define REG_AFO_ERR_STAT		0x136c
> +/* LCSO error status register */
> +#define REG_LCSO_ERR_STAT		0x1370
> +/* BPCI error status register */
> +#define REG_BPCI_ERR_STAT		0x137c
> +/* LSCI error status register */
> +#define REG_LSCI_ERR_STAT		0x1384
> +/* LMVO error status register */
> +#define REG_LMVO_ERR_STAT		0x1390
> +/* FLKO error status register */
> +#define REG_FLKO_ERR_STAT		0x1394
> +/* PSO error status register */
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* CQ0 base address */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +/* Frame sequence number */
> +#define REG_FRAME_SEQ_NUM		0x13b8
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +

I would add a common prefix all the registers in the file.

Also, add some docs to know what those acronyms means would be nice.

> +#endif	/* __MTK_CAM_REGS_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..23fdb8b4abc5
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,2087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-dma-contig.h>

Please sort in alphabetical order.

> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_PSO		BIT(10)

It would be nice to have better names of docs of what these means.

> +
> +#define MTK_ISP_ONE_PIXEL_MODE		1
> +#define MTK_ISP_MIN_RESIZE_RATIO	6
> +#define MTK_ISP_MAX_RUNNING_JOBS	3
> +
> +#define MTK_CAM_CIO_PAD_SRC		4
> +#define MTK_CAM_CIO_PAD_SINK		11
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)

no need for the underscore in __vq

> +{
> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_request *
> +mtk_cam_req_to_dev_req(struct media_request *__req)
> +{
> +	return container_of(__req, struct mtk_cam_dev_request, req);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> +				 struct mtk_cam_dev_request *req,
> +				 enum vb2_buffer_state state)
> +{
> +	struct media_request_object *obj, *obj_prev;
> +	unsigned long flags;
> +	u64 ts_eof = ktime_get_boottime_ns();
> +
> +	if (!cam->streaming)

s/streaming/is_streaming

this makes a bit more intuitive of what the the boolean means.

> +		return;
> +
> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> +
> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> +		struct vb2_buffer *vb;
> +		struct mtk_cam_dev_buffer *buf;
> +		struct mtk_cam_video_device *node;
> +
> +		if (!vb2_request_object_is_buffer(obj))
> +			continue;
> +		vb = container_of(obj, struct vb2_buffer, req_obj);
> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +		spin_lock_irqsave(&node->buf_list_lock, flags);
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> +			vb->timestamp = ts_eof;
> +		else
> +			vb->timestamp = req->timestamp;
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +			return req;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +
> +	return NULL;
> +}
> +
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> +				   unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> +			list_del(&req->list);
> +			break;
> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> +				 req->frame_params.frame_seq_no);

maybe a counter in debugfs instead of the warning.

> +			list_del(&req->list);
> +		} else {
> +			break;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	if (!cam->streaming) {
> +		dev_dbg(cam->dev, "stream is off\n");
> +		return;
> +	}
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> +			dev_dbg(cam->dev, "jobs are full\n");
> +			break;
> +		}
> +		cam->running_job_count++;
> +		list_del(&req->list);
> +		list_add_tail(&req->list, &cam->running_job_list);

list_move_tail() can be used.

> +		mtk_isp_req_enqueue(cam, req);
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> +					       media_dev);
> +	unsigned long flags;
> +
> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> +	vb2_request_queue(req);
> +
> +	/* add to pending job list */
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	mtk_cam_dev_req_try_queue(cam);
> +}
> +
> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}

which patchset are these pixel formats defined?
I couldn't find them in the ones you pointed.

I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
so you can use v4l2_format_info() to get the number of bytes.

> +
> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> +			     struct v4l2_pix_format_mplane *mp)
> +{
> +	unsigned int bpl, ppl;

bytes per line and pixels per line right?

> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);

wouldn't be easier a get_pixel_bytes() function instead of bits?

> +	unsigned int width = mp->width;
> +
> +	bpl = 0;
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		/* Bayer encoding format & 2 bytes alignment */
> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		/*
> +		 * The FULL-G encoding format
> +		 * 1 G component per pixel
> +		 * 1 R component per 4 pixel
> +		 * 1 B component per 4 pixel
> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> +		 */
> +		ppl = DIV_ROUND_UP(width * 6, 4);
> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> +
> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> +		if (pixel_bits == 10)
> +			bpl = ALIGN(bpl, 4);
> +		else
> +			bpl = ALIGN(bpl, 8);
> +	}
> +	/*
> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> +	 */
> +	bpl = ALIGN(bpl, 4);
> +
> +	mp->plane_fmt[0].bytesperline = bpl;
> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> +
> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	int i;

unsigned

> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (!queue_desc->image) {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +		return;
> +	}
> +
> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> +	dest->fmt.pix_mp.num_planes = 1;
> +
> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_R;
> +	default:
> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}

I was wondering if it is not better to save all the media bus format
into a table, instead of having several swtch case statements.

> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}> +}

same for the pixelformat.

Then you can cache object with the pixelformat in the main struct.

> +
> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> +			  struct p1_img_output *out_fmt, int sd_width,
> +			  int sd_height)
> +{
> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> +
> +	/* Check output & input image size dimension */
> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> +			node_id);
> +		return -EINVAL;
> +	}
> +
> +	/* Check resize ratio for resize out stream due to HW constraint */
> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> +		return -EINVAL;
> +	}
> +
> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    !out_fmt->pixel_bits) {
> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> +		return -EINVAL;
> +	}
> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> +
> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +	out_fmt->crop.left = 0;
> +	out_fmt->crop.top = 0;
> +	out_fmt->crop.width = sd_width;
> +	out_fmt->crop.height = sd_height;
> +
> +	dev_dbg(cam->dev,
> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		node_id, out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	cam->enabled_count = 0;
> +	cam->enabled_dmas = 0;
> +	cam->stream_count = 0;
> +	cam->running_job_count = 0;
> +
> +	/* Get the enabled meta DMA ports */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		if (!cam->vdev_nodes[i].enabled)
> +			continue;
> +		cam->enabled_count++;
> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> +	}
> +
> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> +		cam->enabled_dmas);
> +}
> +
> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct p1_config_param config_param;
> +	struct cfg_in_param *cfg_in_param;
> +	struct v4l2_subdev_format sd_fmt;
> +	int sd_width, sd_height, sd_code;

are this sd_* variables required? Can't sd_fmt be directly accessed?

> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> +	int ret;
> +
> +	/* Get sensor format configuration */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> +		return ret;
> +	}
> +	sd_width = sd_fmt.format.width;
> +	sd_height = sd_fmt.format.height;
> +	sd_code = sd_fmt.format.code;
> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> +		sd_code);

If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
it should be configured from userspace.

> +
> +	memset(&config_param, 0, sizeof(config_param));
> +
> +	/* Update cfg_in_param */
> +	cfg_in_param = &config_param.cfg_in_param;
> +	cfg_in_param->continuous = true;
> +	/* Fix to one pixel mode in default */
> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> +	cfg_in_param->crop.width = sd_width;
> +	cfg_in_param->crop.height = sd_height;
> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> +		return -EINVAL;
> +	}
> +
> +	/* Update cfg_main_param */
> +	config_param.cfg_main_param.pure_raw = true;
> +	config_param.cfg_main_param.pure_raw_pack = true;
> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> +			     &config_param.cfg_main_param.output,
> +			     sd_width, sd_height);
> +	if (ret)
> +		return ret;
> +
> +	/* Update cfg_resize_param */
> +	if (enabled_dma_ports & R_RRZO) {
> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> +				     &config_param.cfg_resize_param.output,
> +				     sd_width, sd_height);
> +		if (ret)
> +			return ret;
> +	} else {
> +		config_param.cfg_resize_param.bypass = true;
> +	}
> +
> +	/* Update enabled_dmas */
> +	config_param.enabled_dmas = enabled_dma_ports;
> +	mtk_isp_hw_config(cam, &config_param);
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> +				  unsigned int frame_seq_no)
> +{
> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = frame_seq_no,
> +	};
> +
> +	v4l2_event_queue(cam->subdev.devnode, &event);
> +}
> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> +{
> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> +	struct device *dev = cam->dev;
> +	struct media_entity *entity;
> +	struct v4l2_subdev *sensor;
> +
> +	sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> +			entity->name, entity->function, entity->stream_count);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "no seninf connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "no seninf connected\n");
> +		return -ENODEV;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> +	if (!cam->sensor)
> +		return -ENODEV;
> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		goto fail_seninf_off;
> +	}
> +
> +	ret = mtk_cam_dev_isp_config(cam);
> +	if (ret)
> +		goto fail_sensor_off;
> +
> +	cam->streaming = true;
> +	mtk_isp_stream(cam, 1);
> +	mtk_cam_dev_req_try_queue(cam);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +fail_seninf_off:
> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		return -EPERM;

Why -EPERM ?

> +	}
> +
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	cam->streaming = false;
> +	mtk_isp_stream(cam, 0);
> +	mtk_isp_hw_release(cam);
> +
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> +
> +	if (enable) {
> +		/* Align vb2_core_streamon design */
> +		if (cam->streaming) {
> +			dev_warn(cam->dev, "already streaming on\n");

I think just dev_dbg is enough.

> +			return 0;
> +		}
> +		return mtk_cam_cio_stream_on(cam);
> +	}
> +
> +	if (!cam->streaming) {
> +		dev_warn(cam->dev, "already streaming off\n");

same here

> +		return 0;
> +	}
> +	return mtk_cam_cio_stream_off(cam);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	/*
> +	 * The video nodes exposed by the driver have pads indexes
> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> +	 */
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> +		cam->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);

Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?

> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;

isn't it an issue if userspace queue two buffers for the same video device in the same request?

vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
will be at req->frame_params.dma_bufs[buf->node_id], no?

Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
buffer?

Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.

> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	dma_addr_t addr;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> +	buf->scp_addr = 0;
> +
> +	/* SCP address is only valid for meta input buffer */
> +	if (!node->desc.smem_alloc)
> +		return 0;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	/* Use coherent address to get iova address */
> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> +		return -EFAULT;
> +	}
> +	buf->scp_addr = buf->daddr;
> +	buf->daddr = addr;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> +		if (vb2_get_plane_payload(vb, 0) != size) {
> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> +				vb2_get_plane_payload(vb, 0), size);
> +			return -EINVAL;
> +		}
> +		return 0;
> +	}
> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +	struct device *dev = cam->dev;
> +
> +	if (!node->desc.smem_alloc)
> +		return;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	dma_unmap_page_attrs(dev, buf->daddr,
> +			     vb->planes[0].length,
> +			     DMA_BIDIRECTIONAL,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +}
> +
> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (node->desc.smem_alloc)
> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {
> +		if (sizes[0] < size || *num_planes != 1)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_ret_buf;
> +	}
> +
> +	mutex_lock(&cam->op_lock);
> +	/* Start streaming of the whole pipeline now*/
> +	if (!cam->pipeline.streaming_count) {

No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.

> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> +		if (ret) {
> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> +			goto fail_unlock;
> +		}
> +		mtk_cam_dev_init_stream(cam);
> +		ret = mtk_isp_hw_init(cam);
> +		if (ret) {
> +			dev_err(dev, "failed to init HW:%d\n", ret);
> +			goto fail_stop_pipeline;
> +		}
> +	}
> +
> +	/* Media links are fixed after media_pipeline_start */
> +	cam->stream_count++;
> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> +		cam->enabled_count);
> +	if (cam->stream_count < cam->enabled_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return 0;
> +	}
> +
> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> +	if (ret)
> +		goto fail_no_stream;
> +	mutex_unlock(&cam->op_lock);
> +
> +	return 0;
> +
> +fail_no_stream:
> +	cam->stream_count--;
> +fail_stop_pipeline:
> +	if (cam->stream_count == 0)
> +		media_pipeline_stop(&node->vdev.entity);
> +fail_unlock:
> +	mutex_unlock(&cam->op_lock);
> +fail_ret_buf:
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +
> +	mutex_lock(&cam->op_lock);
> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> +		cam->stream_count);
> +	/* Check the first node to stream-off */
> +	if (cam->stream_count == cam->enabled_count)
> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> +
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> +	cam->stream_count--;
> +	if (cam->stream_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return;
> +	}
> +	mutex_unlock(&cam->op_lock);
> +
> +	mtk_cam_dev_req_cleanup(cam);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +
> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam->dev));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct device *dev = cam->dev;
> +	const struct v4l2_format *dev_fmt;
> +	struct v4l2_format try_fmt;
> +
> +	memset(&try_fmt, 0, sizeof(try_fmt));
> +	try_fmt.type = f->type;
> +
> +	/* Validate pixelformat */
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> +	if (!dev_fmt) {
> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> +	}
> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> +
> +	/* Validate image width & height range */
> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> +	/* 4 bytes alignment for width */
> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> +
> +	/* Only support one plane */
> +	try_fmt.fmt.pix_mp.num_planes = 1;
> +
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> +
> +	/* Constant format fields */
> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +
> +	*f = try_fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (vb2_is_busy(node->vdev.queue)) {
> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> +		return -EBUSY;
> +	}
> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +	/* Configure to video device */
> +	node->vdev_fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};

hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
I wonder if pad ops shouldn't be implemented too (to be verified).

> +
> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> +	.buf_request_complete = mtk_cam_vb2_request_complete,
> +};> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_ops = {
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> +				  struct media_device *media_dev)
> +{
> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	media_dev->dev = cam->dev;
> +	strscpy(media_dev->model, dev_driver_string(dev),
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_ops;
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize subdev pads */
> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> +					sizeof(*cam->subdev_pads),
> +					GFP_KERNEL);
> +	if (!cam->subdev_pads) {
> +		dev_err(dev, "failed to allocate subdev_pads\n");
> +		ret = -ENOMEM;
> +		goto fail_media_unreg;
> +	}
> +
> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> +				     cam->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_pads; i++)
> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	return 0;
> +
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int
> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> +			      struct mtk_cam_video_device *node)
> +{
> +	struct device *dev = cam->dev;
> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> +	unsigned int link_flags = node->desc.link_flags;
> +	int ret;
> +
> +	/* Initialize mtk_cam_video_device */
> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> +		node->enabled = true;
> +	else
> +		node->enabled = false;
> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> +
> +	cam->subdev_pads[node->id].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize vbq */
> +	vbq->type = node->desc.buf_type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam->smem_dev;
> +	} else {
> +		vbq->dev = dev;
> +	}
> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> +	if (output)
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> +	else
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> +	/* No minimum buffers limitation */
> +	vbq->min_buffers_needed = 0;
> +	vbq->drv_priv = cam;
> +	vbq->lock = &node->vdev_lock;
> +	vbq->supports_requests = true;
> +	vbq->requires_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_media_clean;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 dev_driver_string(dev), node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->vdev_lock;
> +	vdev->v4l2_dev = &cam->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> +	vdev->entity.ops = NULL;
> +	video_set_drvdata(vdev, cam);
> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->vdev_lock);
> +	INIT_LIST_HEAD(&node->buf_list);
> +	spin_lock_init(&node->buf_list_lock);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vb2_rel;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam->subdev.entity,
> +					    node->id, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam->subdev.entity,
> +					    node->id, &vdev->entity, 0,
> +					    link_flags);
> +	}

No need for the curly braces.

> +	if (ret)
> +		goto fail_vdev_ureg;
> +
> +	return 0;
> +
> +fail_vdev_ureg:
> +	video_unregister_device(vdev);
> +fail_vb2_rel:
> +	mutex_destroy(&node->vdev_lock);
> +	vb2_queue_release(vbq);
> +fail_media_clean:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static void
> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> +{
> +	video_unregister_device(&node->vdev);
> +	vb2_queue_release(&node->vbq);
> +	media_entity_cleanup(&node->vdev.entity);
> +	mutex_destroy(&node->vdev_lock);
> +}
> +
> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	/* Set up media device & pads */
> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> +	if (ret)
> +		return ret;
> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> +
> +	/* Set up v4l2 device */
> +	cam->v4l2_dev.mdev = &cam->media_dev;
> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> +		 "%s", dev_driver_string(dev));
> +	v4l2_set_subdevdata(&cam->subdev, cam);
> +
> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> +		goto fail_clean_media_entiy;
> +	}
> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> +
> +		node->id = node->desc.id;
> +		ret = mtk_cam_video_register_device(cam, node);
> +		if (ret)
> +			goto fail_vdev_unreg;
> +	}
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_vdev_unreg:
> +	for (i--; i >= 0; i--)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +fail_clean_media_entiy:
> +	media_entity_cleanup(&cam->subdev.entity);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +
> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> +	v4l2_device_unregister_subdev(&cam->subdev);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +	media_entity_cleanup(&cam->subdev.entity);
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> +		return -ENODEV;
> +	}
> +
> +	cam->seninf = sd;
> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam->seninf = NULL;
> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "No seninf subdev\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> +				    MEDIA_LNK_FL_IMMUTABLE |
> +				    MEDIA_LNK_FL_ENABLED);
> +	if (ret) {
> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> +			cam->seninf->entity.name, cam->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	v4l2_async_notifier_init(&cam->notifier);
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);

It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/

Regards,
Helen

> +	if (ret) {
> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> +		return ret;
> +	}
> +
> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> +	if (ret) {
> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> +{
> +	v4l2_async_notifier_unregister(&cam->notifier);
> +	v4l2_async_notifier_cleanup(&cam->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 512 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 1200 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 640 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 288 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	/* This is a default image format */
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> +{
> +	unsigned int node_idx;
> +	int i;
> +
> +	node_idx = 0;
> +	/* Setup the output queue */
> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam)
> +{
> +	int ret;
> +
> +	cam->dev = &pdev->dev;
> +	mtk_cam_dev_queue_setup(cam);
> +
> +	spin_lock_init(&cam->pending_job_lock);
> +	spin_lock_init(&cam->running_job_lock);
> +	INIT_LIST_HEAD(&cam->pending_job_list);
> +	INIT_LIST_HEAD(&cam->running_job_list);
> +	mutex_init(&cam->op_lock);
> +
> +	/* v4l2 sub-device registration */
> +	ret = mtk_cam_v4l2_register(cam);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam);
> +	if (ret)
> +		goto fail_v4l2_unreg;
> +
> +	return 0;
> +
> +fail_v4l2_unreg:
> +	mutex_destroy(&cam->op_lock);
> +	mtk_cam_v4l2_unregister(cam);
> +
> +	return ret;
> +}
> +
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> +{
> +	mtk_cam_v4l2_async_unregister(cam);
> +	mtk_cam_v4l2_unregister(cam);
> +	mutex_destroy(&cam->op_lock);
> +}
> +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..0a340a1e65ea
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,244 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_H__
> +#define __MTK_CAM_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "mtk_cam-ipi.h"
> +
> +#define IMG_MAX_WIDTH		5376
> +#define IMG_MAX_HEIGHT		4032
> +#define IMG_MIN_WIDTH		80
> +#define IMG_MIN_HEIGHT		60
> +
> +/*
> + * ID enum value for struct mtk_cam_dev_node_desc:id
> + * or mtk_cam_video_device:id
> + */
> +enum  {
> +	MTK_CAM_P1_META_IN_0 = 0,
> +	MTK_CAM_P1_MAIN_STREAM_OUT,
> +	MTK_CAM_P1_PACKED_BIN_OUT,
> +	MTK_CAM_P1_META_OUT_0,
> +	MTK_CAM_P1_META_OUT_1,
> +	MTK_CAM_P1_META_OUT_2,
> +	MTK_CAM_P1_META_OUT_3,
> +	MTK_CAM_P1_TOTAL_NODES
> +};
> +
> +/* Supported image format list */
> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> +
> +/* Supported bayer pixel order */
> +#define MTK_CAM_RAW_PXL_ID_B		0
> +#define MTK_CAM_RAW_PXL_ID_GB		1
> +#define MTK_CAM_RAW_PXL_ID_GR		2
> +#define MTK_CAM_RAW_PXL_ID_R		3
> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> +
> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +/*
> + * struct mtk_cam_dev_request - MTK camera device request.
> + *
> + * @req: Embedded struct media request.
> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> + * @frame_work: work queue entry for frame transmission to SCP.
> + * @list: List entry of the object for @struct mtk_cam_dev:
> + *        pending_job_list or running_job_list.
> + * @timestamp: Start of frame timestamp in ns
> + *
> + */
> +struct mtk_cam_dev_request {
> +	struct media_request req;
> +	struct mtk_p1_frame_param frame_params;
> +	struct work_struct frame_work;
> +	struct list_head list;
> +	u64 timestamp;
> +};
> +
> +/*
> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> + *
> + * @vbb: Embedded struct vb2_v4l2_buffer.
> + * @list: List entry of the object for @struct mtk_cam_video_device:
> + *        buf_list.
> + * @daddr: The DMA address of this buffer.
> + * @scp_addr: The SCP address of this buffer which
> + *            is only supported for meta input node.
> + * @node_id: The vidoe node id which this buffer belongs to.
> + *
> + */
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer vbb;
> +	struct list_head list;
> +	/* Intenal part */
> +	dma_addr_t daddr;
> +	dma_addr_t scp_addr;
> +	unsigned int node_id;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> + *
> + * @id: id of the node
> + * @name: name of the node
> + * @cap: supported V4L2 capabilities
> + * @buf_type: supported V4L2 buffer type
> + * @dma_port: the dma ports associated to the node
> + * @link_flags: default media link flags
> + * @smem_alloc: using the smem_dev as alloc device or not
> + * @image: true for image node, false for meta node
> + * @num_fmts: the number of supported node formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum VB2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts: supported format
> + * @frmsizes: supported V4L2 frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	const char *name;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure
> + *
> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> + * @enabled: Indicate the video device is enabled or not
> + * @desc: The node description of video device
> + * @vdev_fmt: The V4L2 format of video device
> + * @vdev_pad: The media pad graph object of video device
> + * @vbq: A videobuf queue of video device
> + * @vdev: The video device instance
> + * @vdev_lock: Serializes vb2 queue and video device operations
> + * @buf_list: List for enqueue buffers
> + * @buf_list_lock: Lock used to protect buffer list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_format vdev_fmt;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct video_device vdev;
> +	/* Serializes vb2 queue and video device operations */
> +	struct mutex vdev_lock;
> +	struct list_head buf_list;
> +	/* Lock used to protect buffer list */
> +	spinlock_t buf_list_lock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @dev: Pointer to device.
> + * @smem_pdev: Pointer to shared memory device.
> + * @pipeline: Media pipeline information.
> + * @media_dev: Media device instance.
> + * @subdev: The V4L2 sub-device instance.
> + * @v4l2_dev: The V4L2 device driver instance.
> + * @notifier: The v4l2_device notifier data.
> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> + * @seninf: Pointer to the seninf sub-device.
> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> + * @streaming: Indicate the overall streaming status is on or off.
> + * @enabled_dmas: The enabled dma port information when streaming on.
> + * @enabled_count: Number of enabled video nodes
> + * @stream_count: Number of streaming video nodes
> + * @running_job_count: Nunber of running jobs in the HW driver.
> + * @pending_job_list: List to keep the media requests before en-queue into
> + *                    HW driver.
> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> + * @running_job_list: List to keep the media requests after en-queue into
> + *                    HW driver.
> + * @running_job_lock: Protect the running_job_list data.
> + * @op_lock: Serializes driver's VB2 callback operations.
> + *
> + */
> +struct mtk_cam_dev {
> +	struct device *dev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int enabled_dmas;
> +	unsigned int enabled_count;
> +	unsigned int stream_count;
> +	unsigned int running_job_count;
> +	struct list_head pending_job_list;
> +	/* Protect the pending_job_list data */
> +	spinlock_t pending_job_lock;
> +	struct list_head running_job_list;
> +	/* Protect the running_job_list data & running_job_count */
> +	spinlock_t running_job_lock;
> +	/* Serializes driver's VB2 callback operations */
> +	struct mutex op_lock;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> +				   unsigned int frame_seq_no);
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				  unsigned int frame_seq_no);
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no);
> +
> +#endif /* __MTK_CAM_H__ */
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2019-12-19  5:49     ` Jungo Lin
@ 2020-04-02 16:45       ` Dafna Hirschfeld
  -1 siblings, 0 replies; 388+ messages in thread
From: Dafna Hirschfeld @ 2020-04-02 16:45 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: linux-media, linux-mediatek, linux-arm-kernel, devicetree,
	srv_heupstream, ddavenport, robh, Sean.Cheng, sj.huang,
	frederic.chen, Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu,
	yuzhao, zwisler, shik, suleiman, Pi-Hsun Shih



On 19.12.19 06:49, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>   - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>   - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>   - Correct auto suspend timer value for suspend/resume issue
>   - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>   - Fix KE due to no sen-inf sub-device
> ---
>   drivers/media/platform/mtk-isp/Kconfig        |   20 +
>   .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>   .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>   .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>   .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>   9 files changed, 3377 insertions(+)
>   create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> new file mode 100644
> index 000000000000..f86e1b59ad1e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/Kconfig
> @@ -0,0 +1,20 @@
> +config VIDEO_MEDIATEK_ISP_PASS1
> +	tristate "Mediatek ISP Pass 1 driver"
> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> +	depends on ARCH_MEDIATEK
> +	select V4L2_FWNODE
> +	select VIDEOBUF2_VMALLOC
> +	select VIDEOBUF2_DMA_CONTIG
> +	select MTK_SCP
> +	default n
> +	help
> +		Pass 1 driver controls 3A (auto-focus, exposure,
> +		and white balance) with tuning feature and outputs
> +		the captured image buffers in Mediatek's camera system.
> +
> +		Choose Y if you want to use Mediatek SoCs to create image
> +		captured application such as video recording and still image
> +		capturing.
> +
> +		To compile this driver as a module, choose M here; the module
> +		will be called mtk-cam-isp.
> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> new file mode 100644
> index 000000000000..ce79d283b209
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..53b54d3c26a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +mtk-cam-isp-objs += mtk_cam.o
> +mtk-cam-isp-objs += mtk_cam-hw.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> new file mode 100644
> index 000000000000..4065d0d29b7f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> @@ -0,0 +1,636 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/module.h>
> +#include <linux/remoteproc/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include <media/v4l2-event.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +#include "mtk_cam-regs.h"
> +
> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> +#define MTK_ISP_CQ_BUFFER_COUNT			3
> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> +
> +/*
> + *
> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> + * In current driver, only supports CAM B.
> + *
> + */
> +#define MTK_ISP_CAM_ID_B			3
> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> +
> +static void isp_tx_frame_worker(struct work_struct *work)
> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> +
> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> +		dev_err(dev, "wrong IPI len:%d\n", len);
> +		return;
> +	}
> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> +		return;
> +
> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> +}
> +
> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI cmd\n");
> +		return ret;
> +	}
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI frame\n");
> +		goto unreg_ipi_cmd;
> +	}
> +
> +	p1_dev->composer_wq =
> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> +					WQ_FREEZABLE);
> +	if (!p1_dev->composer_wq) {
> +		dev_err(dev, "failed to alloc composer workqueue\n");
> +		goto unreg_ipi_frame;
> +	}
> +
> +	return 0;
> +
> +unreg_ipi_frame:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +unreg_ipi_cmd:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +
> +	return ret;
> +}
> +
> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	destroy_workqueue(p1_dev->composer_wq);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +}
> +
> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> +
> +	/*
> +	 * Passed coherent reserved memory info. for SCP firmware usage.
> +	 * This buffer is used for SCP's ISP composer to compose.
> +	 * The size of is fixed to 0x200000 for the requirement of composer.
> +	 */
> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +
> +	isp_composer_uninit(p1_dev);
> +}
> +
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> +		       struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		dev_err(dev, "failed to rproc_boot\n");
> +		return ret;
> +	}
> +
> +	ret = isp_composer_init(p1_dev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_get_sync(dev);
> +	isp_composer_hw_init(p1_dev);
> +
> +	p1_dev->enqueued_frame_seq_no = 0;
> +	p1_dev->dequeued_frame_seq_no = 0;
> +	p1_dev->composed_frame_seq_no = 0;
> +	p1_dev->sof_count = 0;
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	isp_composer_hw_deinit(p1_dev);
> +	pm_runtime_mark_last_busy(dev);
> +	pm_runtime_put_autosuspend(dev);
> +	rproc_shutdown(p1_dev->rproc_handle);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}
> +
> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> +			       unsigned int dequeued_frame_seq_no)
> +{
> +	dma_addr_t base_addr = p1_dev->composer_iova;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_cam_dev_request *req;
> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> +	unsigned int addr_offset;
> +
> +	/* Send V4L2_EVENT_FRAME_SYNC event */
> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +
> +	p1_dev->sof_count += 1;
> +	/* Save frame information */
> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> +
> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +	if (req)
> +		req->timestamp = ktime_get_boottime_ns();
> +
> +	/* Update CQ base address if needed */
> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> +		dev_dbg(dev,
> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> +			composed_frame_seq_no, dequeued_frame_seq_no);
> +		return;
> +	}
> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> +}
> +
> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> +{
> +	u32 val;
> +
> +	dev_err(p1_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(p1_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> +
> +	/* Disable DMA error mask to avoid too much error log */
> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> +}
> +
> +static irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> +	struct device *dev = p1_dev->dev;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> +
> +	/*
> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> +	 * If these two ISRs come together, print warning msg to hint.
> +	 */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> +
> +	/* De-queue frame */
> +	if (irq_status & SW_PASS1_DON_ST) {
> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> +					      p1_dev->dequeued_frame_seq_no);
> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> +	}
> +
> +	/* Save frame info. & update CQ address for frame HW en-queue */
> +	if (irq_status & SOF_INT_ST)
> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_irq_handle_dma_err(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> +		p1_dev->sof_count, irq_status, dma_status,
> +		dequeued_frame_seq_no);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> +			       struct platform_device *pdev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	dma_addr_t addr;
> +	void *ptr;
> +	int ret;
> +
> +	p1_dev->scp = scp_get(pdev);
> +	if (!p1_dev->scp) {
> +		dev_err(dev, "failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> +
> +	/*
> +	 * Allocate coherent reserved memory for SCP firmware usage.
> +	 * The size of SCP composer's memory is fixed to 0x200000
> +	 * for the requirement of firmware.
> +	 */
> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> +	if (!ptr) {
> +		ret = -ENOMEM;
> +		goto fail_put_scp;
> +	}
> +
> +	p1_dev->composer_scp_addr = addr;
> +	p1_dev->composer_virt_addr = ptr;
> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> +
> +	/*
> +	 * This reserved memory is also be used by ISP P1 HW.
> +	 * Need to get iova address for ISP P1 DMA.
> +	 */
> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map scp iova\n");
> +		ret = -ENOMEM;
> +		goto fail_free_mem;
> +	}
> +	p1_dev->composer_iova = addr;
> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> +
> +	return 0;
> +
> +fail_free_mem:
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +fail_put_scp:
> +	scp_put(p1_dev->scp);
> +
> +	return ret;
> +}
> +
> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> +{
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +	scp_put(p1_dev->scp);
> +}
> +
> +static int mtk_isp_pm_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Disable ISP's view finder and wait for TG idle if possible */
> +	dev_dbg(dev, "cam suspend, disable VF\n");
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> +
> +	/* Disable CMOS */
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Force ISP HW to idle */
> +	ret = pm_runtime_force_suspend(dev);
> +	if (ret) {
> +		dev_err(dev, "failed to force suspend:%d\n", ret);
> +		goto reenable_hw;
> +	}
> +
> +	return 0;
> +
> +reenable_hw:
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_pm_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Force ISP HW to resume */
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret)
> +		return ret;
> +
> +	/* Enable CMOS */
> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Enable VF */
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	dev_dbg(dev, "%s:disable clock\n", __func__);
> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	dev_dbg(dev, "%s:enable clock\n", __func__);
> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to enable clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	/* List of clocks required by isp cam */
> +	static const char * const clk_names[] = {
> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +	};
> +	struct mtk_isp_p1_device *p1_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq, ret, i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	p1_dev->dev = dev;
> +	dev_set_drvdata(dev, p1_dev);
> +
> +	/*
> +	 * Now only support single CAM with CAM B.
> +	 * Get CAM B register base with CAM B index.
> +	 * Support multiple CAMs in future.
> +	 */
> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> +	p1_dev->regs = devm_ioremap_resource(dev, res);
> +	if (IS_ERR(p1_dev->regs)) {
> +		dev_err(dev, "failed to map reister base\n");
> +		return PTR_ERR(p1_dev->regs);
> +	}
> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> +
> +	/*
> +	 * The cam_sys unit only supports reg., but has no IRQ support.
> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> +	 */
> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> +	if (!irq) {
> +		dev_err(dev, "failed to get irq\n");
> +		return -ENODEV;
> +	}
> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> +			       p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to request irq=%d\n", irq);
> +		return ret;
> +	}
> +	dev_dbg(dev, "registered irq=%d\n", irq);
> +	spin_lock_init(&p1_dev->spinlock_irq);
> +
> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> +	if (!p1_dev->clks)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->num_clks; ++i)
> +		p1_dev->clks[i].id = clk_names[i];
> +
> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> +	pm_runtime_use_autosuspend(dev);
> +	pm_runtime_enable(dev);
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret) {
> +		isp_teardown_scp_rproc(p1_dev);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> +	pm_runtime_dont_use_autosuspend(dev);
> +	pm_runtime_disable(dev);
> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +	isp_teardown_scp_rproc(p1_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> +			   NULL)
> +};
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam-p1",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> new file mode 100644
> index 000000000000..837662f92a5e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> @@ -0,0 +1,64 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_HW_H__
> +#define __MTK_CAM_HW_H__
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ipi.h"
> +
> +/*
> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> + *
> + * @dev: Pointer to device.
> + * @scp_pdev: Pointer to SCP platform device.
> + * @rproc_handle: Pointer to new remoteproc instance.
> + * @cam_dev: Embedded struct cam_dev
> + * @regs: Camera ISP HW base register address
> + * @num_clks: The number of driver's clocks
> + * @clks: The clock data array
> + * @spinlock_irq: Used to protect register read/write data
> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> + * @composed_frame_seq_no: Frame sequence number of composed frame
> + * @timestamp: Frame timestamp in ns
> + * @sof_count: SOF counter
> + * @composer_wq: The work queue for frame request composing
> + * @composer_scp_addr: SCP address of ISP composer memory
> + * @composer_iova: DMA address of ISP composer memory
> + * @virt_addr: Virtual address of ISP composer memory
> + *
> + */
> +struct mtk_isp_p1_device {
> +	struct device *dev;
> +	struct mtk_scp *scp;
> +	struct rproc *rproc_handle;
> +	struct mtk_cam_dev cam_dev;
> +	void __iomem *regs;
> +	unsigned int num_clks;
> +	struct clk_bulk_data *clks;
> +	/* Used to protect register read/write data */
> +	spinlock_t spinlock_irq;
> +	unsigned int enqueued_frame_seq_no;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int composed_frame_seq_no;
> +	u8 sof_count;
> +	struct workqueue_struct *composer_wq;
> +	dma_addr_t composer_scp_addr;
> +	dma_addr_t composer_iova;
> +	void *composer_virt_addr;
> +};
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> +		       struct p1_config_param *config_param);
> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> +			 struct mtk_cam_dev_request *req);
> +
> +#endif /* __MTK_CAM_HW_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> new file mode 100644
> index 000000000000..981b634dd91f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> @@ -0,0 +1,222 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_IPI_H__
> +#define __MTK_CAM_IPI_H__
> +
> +#include <linux/types.h>
> +
> +/*
> + * struct img_size - Image size information.
> + *
> + * @w: Image width, the unit is pixel
> + * @h: Image height, the unit is pixel
> + * @xsize: Bytes per line based on width.
> + * @stride: Bytes per line when changing line.
> + *          Stride is based on xsize + HW constrain(byte align).
> + *
> + */
> +struct img_size {
> +	u32 w;
> +	u32 h;
> +	u32 xsize;
> +	u32 stride;
> +} __packed;
> +
> +/*
> + * struct p1_img_crop - image corp information
> + *
> + * @left: The left of crop area.
> + * @top: The top of crop area.
> + * @width: The width of crop area.
> + * @height: The height of crop area.
> + *
> + */
> +struct p1_img_crop {
> +	u32 left;
> +	u32 top;
> +	u32 width;
> +	u32 height;
> +} __packed;
> +
> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;
> +	u32 scp_addr;
> +} __packed;
> +
> +/*
> + * struct p1_img_output - ISP P1 image output information
> + *
> + * @buffer: DMA buffer address of image.
> + * @size: The image size configuration.
> + * @crop: The crop configuration.
> + * @pixel_bits: The bits per image pixel.
> + * @img_fmt: The image format.
> + *
> + */
> +struct p1_img_output {
> +	struct dma_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	u8 pixel_bits;
> +	u32 img_fmt;
> +} __packed;
> +
> +/*
> + * struct cfg_in_param - Image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: Indicate the sensor mode. Continuous or single shot.
> + * @subsample: Indicate to enables SOF subsample or not.
> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: Describe input data pattern.
> + * @raw_pixel_id: Bayer sequence.
> + * @tg_fps: The fps rate of TG (time generator).
> + * @img_fmt: The image format of input source.
> + * @p1_img_crop: The crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	u8 continuous;
> +	u8 subsample;
> +	u8 pixel_mode;
> +	u8 data_pattern;
> +	u8 raw_pixel_id;
> +	u16 tg_fps;
> +	u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - The image output parameters of main stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @pure_raw: Indicate the image path control.
> + *            True: pure raw
> + *            False: processing raw
> + * @pure_raw_pack: Indicate the image is packed or not.
> + *                 True: packed mode
> + *                 False: unpacked mode
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	u8 bypass;
> +	u8 pure_raw;
> +	u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - The image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct p1_config_param - ISP P1 configuration parameters.
> + *
> + * @cfg_in_param: The Image input parameters.
> + * @cfg_main_param: The main output image parameters.
> + * @cfg_resize_out_param: The packed output image parameters.
> + * @enabled_dmas: The enabled DMA port information.
> + *
> + */
> +struct p1_config_param {
> +	struct cfg_in_param cfg_in_param;
> +	struct cfg_main_out_param cfg_main_param;
> +	struct cfg_resize_out_param cfg_resize_param;
> +	u32 enabled_dmas;
> +} __packed;
> +
> +/*
> + * struct P1_meta_frame - ISP P1 meta frame information.
> + *
> + * @enabled_dma: The enabled DMA port information.
> + * @vb_index: The VB2 index of meta buffer.
> + * @meta_addr: DMA buffer address of meta buffer.
> + *
> + */
> +struct P1_meta_frame {
> +	u32 enabled_dma;
> +	u32 vb_index;
> +	struct dma_buffer meta_addr;
> +} __packed;
> +
> +/*
> + * struct isp_init_info - ISP P1 composer init information.
> + *
> + * @hw_module: The ISP Camera HW module ID.
> + * @cq_addr: The DMA address of composer memory.
> + *
> + */
> +struct isp_init_info {
> +	u8 hw_module;
> +	struct dma_buffer cq_addr;
> +} __packed;
> +
> +/*
> + * struct isp_ack_info - ISP P1 IPI command ack information.
> + *
> + * @cmd_id: The IPI command ID is acked.
> + * @frame_seq_no: The IPI frame sequence number is acked.
> + *
> + */
> +struct isp_ack_info {
> +	u8 cmd_id;
> +	u32 frame_seq_no;
> +} __packed;
> +
> +/*
> + * The IPI command enumeration.
> + */
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_RESERVED,
> +};
> +
> +/*
> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> + *
> + * @cmd_id: The IPI command ID.
> + * @init_param: The init formation for ISP_CMD_INIT.
> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> + *
> + */
> +struct mtk_isp_scp_p1_cmd {
> +	u8 cmd_id;
> +	union {
> +		struct isp_init_info init_param;
> +		struct p1_config_param config_param;
> +		u32 enabled_dmas;
> +		struct P1_meta_frame meta_frame;
> +		u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +#endif /* __MTK_CAM_IPI_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..ab2277f45fa4
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,95 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_REGS_H__
> +#define __MTK_CAM_REGS_H__
> +
> +/* ISP interrupt enable */
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define DMA_ERR_INT_EN			BIT(29)
> +
> +/* ISP interrupt status */
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define VS_INT_ST			BIT(0)
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define HW_PASS1_DON_ST			BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +#define SW_PASS1_DON_ST			BIT(30)
> +
> +/* ISP interrupt 2 status */
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +
> +/* Configures sensor mode */
> +#define REG_TG_SEN_MODE			0x0230
> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_VF_CON			0x0234
> +#define TG_VF_CON_VFDATA_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_INTER_ST			0x026c
> +#define TG_CS_MASK			0x3f00
> +#define TG_IDLE_ST			BIT(8)
> +
> +/* IMGO error status register */
> +#define REG_IMGO_ERR_STAT		0x1360
> +/* RRZO error status register */
> +#define REG_RRZO_ERR_STAT		0x1364
> +/* AAO error status register */
> +#define REG_AAO_ERR_STAT		0x1368
> +/* AFO error status register */
> +#define REG_AFO_ERR_STAT		0x136c
> +/* LCSO error status register */
> +#define REG_LCSO_ERR_STAT		0x1370
> +/* BPCI error status register */
> +#define REG_BPCI_ERR_STAT		0x137c
> +/* LSCI error status register */
> +#define REG_LSCI_ERR_STAT		0x1384
> +/* LMVO error status register */
> +#define REG_LMVO_ERR_STAT		0x1390
> +/* FLKO error status register */
> +#define REG_FLKO_ERR_STAT		0x1394
> +/* PSO error status register */
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* CQ0 base address */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +/* Frame sequence number */
> +#define REG_FRAME_SEQ_NUM		0x13b8
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +#endif	/* __MTK_CAM_REGS_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..23fdb8b4abc5
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,2087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_PSO		BIT(10)
> +
> +#define MTK_ISP_ONE_PIXEL_MODE		1
> +#define MTK_ISP_MIN_RESIZE_RATIO	6
> +#define MTK_ISP_MAX_RUNNING_JOBS	3
> +
> +#define MTK_CAM_CIO_PAD_SRC		4
> +#define MTK_CAM_CIO_PAD_SINK		11
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> +{
> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_request *
> +mtk_cam_req_to_dev_req(struct media_request *__req)
> +{
> +	return container_of(__req, struct mtk_cam_dev_request, req);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> +				 struct mtk_cam_dev_request *req,
> +				 enum vb2_buffer_state state)
> +{
> +	struct media_request_object *obj, *obj_prev;
> +	unsigned long flags;
> +	u64 ts_eof = ktime_get_boottime_ns();
> +
> +	if (!cam->streaming)
> +		return;
> +
> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> +
> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> +		struct vb2_buffer *vb;
> +		struct mtk_cam_dev_buffer *buf;
> +		struct mtk_cam_video_device *node;
> +
> +		if (!vb2_request_object_is_buffer(obj))
> +			continue;
> +		vb = container_of(obj, struct vb2_buffer, req_obj);
> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +		spin_lock_irqsave(&node->buf_list_lock, flags);
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> +			vb->timestamp = ts_eof;
> +		else
> +			vb->timestamp = req->timestamp;
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +			return req;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +
> +	return NULL;
> +}
> +
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> +				   unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> +			list_del(&req->list);
> +			break;
> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
Hi, I see that frame_params.frame_seq_no is incremented when a request is queued
and frame_seq_no is read from a register, so if the first is lower than the latter
it means userspace was queueing request too slowly and missed frames right?
So userspace will have to catch up in order not to get "ERROR" buffers.
Maybe the driver can just skip lost frames. In the rkisp1 for example,
if there is no buffer available, the driver just write to a dummy buffer that is
never sent to userspace. This way userspace always get valid buffers back when
dequeueing.

> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> +				 req->frame_params.frame_seq_no);
> +			list_del(&req->list);
> +		} else {
> +			break;
Does this case can ever occur?

Thanks,
Dafna

> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	if (!cam->streaming) {
> +		dev_dbg(cam->dev, "stream is off\n");
> +		return;
> +	}
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> +			dev_dbg(cam->dev, "jobs are full\n");
> +			break;
> +		}
> +		cam->running_job_count++;
> +		list_del(&req->list);
> +		list_add_tail(&req->list, &cam->running_job_list);
> +		mtk_isp_req_enqueue(cam, req);
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> +					       media_dev);
> +	unsigned long flags;
> +
> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> +	vb2_request_queue(req);
> +
> +	/* add to pending job list */
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	mtk_cam_dev_req_try_queue(cam);
> +}
> +
> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> +			     struct v4l2_pix_format_mplane *mp)
> +{
> +	unsigned int bpl, ppl;
> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> +	unsigned int width = mp->width;
> +
> +	bpl = 0;
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		/* Bayer encoding format & 2 bytes alignment */
> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		/*
> +		 * The FULL-G encoding format
> +		 * 1 G component per pixel
> +		 * 1 R component per 4 pixel
> +		 * 1 B component per 4 pixel
> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> +		 */
> +		ppl = DIV_ROUND_UP(width * 6, 4);
> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> +
> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> +		if (pixel_bits == 10)
> +			bpl = ALIGN(bpl, 4);
> +		else
> +			bpl = ALIGN(bpl, 8);
> +	}
> +	/*
> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> +	 */
> +	bpl = ALIGN(bpl, 4);
> +
> +	mp->plane_fmt[0].bytesperline = bpl;
> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> +
> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	int i;
> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (!queue_desc->image) {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +		return;
> +	}
> +
> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> +	dest->fmt.pix_mp.num_planes = 1;
> +
> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_R;
> +	default:
> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> +			  struct p1_img_output *out_fmt, int sd_width,
> +			  int sd_height)
> +{
> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> +
> +	/* Check output & input image size dimension */
> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> +			node_id);
> +		return -EINVAL;
> +	}
> +
> +	/* Check resize ratio for resize out stream due to HW constraint */
> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> +		return -EINVAL;
> +	}
> +
> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    !out_fmt->pixel_bits) {
> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> +		return -EINVAL;
> +	}
> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> +
> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +	out_fmt->crop.left = 0;
> +	out_fmt->crop.top = 0;
> +	out_fmt->crop.width = sd_width;
> +	out_fmt->crop.height = sd_height;
> +
> +	dev_dbg(cam->dev,
> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		node_id, out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	cam->enabled_count = 0;
> +	cam->enabled_dmas = 0;
> +	cam->stream_count = 0;
> +	cam->running_job_count = 0;
> +
> +	/* Get the enabled meta DMA ports */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		if (!cam->vdev_nodes[i].enabled)
> +			continue;
> +		cam->enabled_count++;
> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> +	}
> +
> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> +		cam->enabled_dmas);
> +}
> +
> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct p1_config_param config_param;
> +	struct cfg_in_param *cfg_in_param;
> +	struct v4l2_subdev_format sd_fmt;
> +	int sd_width, sd_height, sd_code;
> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> +	int ret;
> +
> +	/* Get sensor format configuration */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> +		return ret;
> +	}
> +	sd_width = sd_fmt.format.width;
> +	sd_height = sd_fmt.format.height;
> +	sd_code = sd_fmt.format.code;
> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> +		sd_code);
> +
> +	memset(&config_param, 0, sizeof(config_param));
> +
> +	/* Update cfg_in_param */
> +	cfg_in_param = &config_param.cfg_in_param;
> +	cfg_in_param->continuous = true;
> +	/* Fix to one pixel mode in default */
> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> +	cfg_in_param->crop.width = sd_width;
> +	cfg_in_param->crop.height = sd_height;
> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> +		return -EINVAL;
> +	}
> +
> +	/* Update cfg_main_param */
> +	config_param.cfg_main_param.pure_raw = true;
> +	config_param.cfg_main_param.pure_raw_pack = true;
> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> +			     &config_param.cfg_main_param.output,
> +			     sd_width, sd_height);
> +	if (ret)
> +		return ret;
> +
> +	/* Update cfg_resize_param */
> +	if (enabled_dma_ports & R_RRZO) {
> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> +				     &config_param.cfg_resize_param.output,
> +				     sd_width, sd_height);
> +		if (ret)
> +			return ret;
> +	} else {
> +		config_param.cfg_resize_param.bypass = true;
> +	}
> +
> +	/* Update enabled_dmas */
> +	config_param.enabled_dmas = enabled_dma_ports;
> +	mtk_isp_hw_config(cam, &config_param);
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> +				  unsigned int frame_seq_no)
> +{
> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = frame_seq_no,
> +	};
> +
> +	v4l2_event_queue(cam->subdev.devnode, &event);
> +}
> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> +{
> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> +	struct device *dev = cam->dev;
> +	struct media_entity *entity;
> +	struct v4l2_subdev *sensor;
> +
> +	sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> +			entity->name, entity->function, entity->stream_count);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "no seninf connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "no seninf connected\n");
> +		return -ENODEV;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> +	if (!cam->sensor)
> +		return -ENODEV;
> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		goto fail_seninf_off;
> +	}
> +
> +	ret = mtk_cam_dev_isp_config(cam);
> +	if (ret)
> +		goto fail_sensor_off;
> +
> +	cam->streaming = true;
> +	mtk_isp_stream(cam, 1);
> +	mtk_cam_dev_req_try_queue(cam);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +fail_seninf_off:
> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	cam->streaming = false;
> +	mtk_isp_stream(cam, 0);
> +	mtk_isp_hw_release(cam);
> +
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> +
> +	if (enable) {
> +		/* Align vb2_core_streamon design */
> +		if (cam->streaming) {
> +			dev_warn(cam->dev, "already streaming on\n");
> +			return 0;
> +		}
> +		return mtk_cam_cio_stream_on(cam);
> +	}
> +
> +	if (!cam->streaming) {
> +		dev_warn(cam->dev, "already streaming off\n");
> +		return 0;
> +	}
> +	return mtk_cam_cio_stream_off(cam);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	/*
> +	 * The video nodes exposed by the driver have pads indexes
> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> +	 */
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> +		cam->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	dma_addr_t addr;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> +	buf->scp_addr = 0;
> +
> +	/* SCP address is only valid for meta input buffer */
> +	if (!node->desc.smem_alloc)
> +		return 0;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	/* Use coherent address to get iova address */
> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> +		return -EFAULT;
> +	}
> +	buf->scp_addr = buf->daddr;
> +	buf->daddr = addr;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> +		if (vb2_get_plane_payload(vb, 0) != size) {
> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> +				vb2_get_plane_payload(vb, 0), size);
> +			return -EINVAL;
> +		}
> +		return 0;
> +	}
> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +	struct device *dev = cam->dev;
> +
> +	if (!node->desc.smem_alloc)
> +		return;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	dma_unmap_page_attrs(dev, buf->daddr,
> +			     vb->planes[0].length,
> +			     DMA_BIDIRECTIONAL,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +}
> +
> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (node->desc.smem_alloc)
> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {
> +		if (sizes[0] < size || *num_planes != 1)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_ret_buf;
> +	}
> +
> +	mutex_lock(&cam->op_lock);
> +	/* Start streaming of the whole pipeline now*/
> +	if (!cam->pipeline.streaming_count) {
> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> +		if (ret) {
> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> +			goto fail_unlock;
> +		}
> +		mtk_cam_dev_init_stream(cam);
> +		ret = mtk_isp_hw_init(cam);
> +		if (ret) {
> +			dev_err(dev, "failed to init HW:%d\n", ret);
> +			goto fail_stop_pipeline;
> +		}
> +	}
> +
> +	/* Media links are fixed after media_pipeline_start */
> +	cam->stream_count++;
> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> +		cam->enabled_count);
> +	if (cam->stream_count < cam->enabled_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return 0;
> +	}
> +
> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> +	if (ret)
> +		goto fail_no_stream;
> +	mutex_unlock(&cam->op_lock);
> +
> +	return 0;
> +
> +fail_no_stream:
> +	cam->stream_count--;
> +fail_stop_pipeline:
> +	if (cam->stream_count == 0)
> +		media_pipeline_stop(&node->vdev.entity);
> +fail_unlock:
> +	mutex_unlock(&cam->op_lock);
> +fail_ret_buf:
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +
> +	mutex_lock(&cam->op_lock);
> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> +		cam->stream_count);
> +	/* Check the first node to stream-off */
> +	if (cam->stream_count == cam->enabled_count)
> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> +
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> +	cam->stream_count--;
> +	if (cam->stream_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return;
> +	}
> +	mutex_unlock(&cam->op_lock);
> +
> +	mtk_cam_dev_req_cleanup(cam);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +
> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam->dev));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct device *dev = cam->dev;
> +	const struct v4l2_format *dev_fmt;
> +	struct v4l2_format try_fmt;
> +
> +	memset(&try_fmt, 0, sizeof(try_fmt));
> +	try_fmt.type = f->type;
> +
> +	/* Validate pixelformat */
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> +	if (!dev_fmt) {
> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> +	}
> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> +
> +	/* Validate image width & height range */
> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> +	/* 4 bytes alignment for width */
> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> +
> +	/* Only support one plane */
> +	try_fmt.fmt.pix_mp.num_planes = 1;
> +
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> +
> +	/* Constant format fields */
> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +
> +	*f = try_fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (vb2_is_busy(node->vdev.queue)) {
> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> +		return -EBUSY;
> +	}
> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +	/* Configure to video device */
> +	node->vdev_fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> +	.buf_request_complete = mtk_cam_vb2_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_ops = {
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> +				  struct media_device *media_dev)
> +{
> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	media_dev->dev = cam->dev;
> +	strscpy(media_dev->model, dev_driver_string(dev),
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_ops;
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize subdev pads */
> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> +					sizeof(*cam->subdev_pads),
> +					GFP_KERNEL);
> +	if (!cam->subdev_pads) {
> +		dev_err(dev, "failed to allocate subdev_pads\n");
> +		ret = -ENOMEM;
> +		goto fail_media_unreg;
> +	}
> +
> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> +				     cam->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_pads; i++)
> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	return 0;
> +
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int
> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> +			      struct mtk_cam_video_device *node)
> +{
> +	struct device *dev = cam->dev;
> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> +	unsigned int link_flags = node->desc.link_flags;
> +	int ret;
> +
> +	/* Initialize mtk_cam_video_device */
> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> +		node->enabled = true;
> +	else
> +		node->enabled = false;
> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> +
> +	cam->subdev_pads[node->id].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize vbq */
> +	vbq->type = node->desc.buf_type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam->smem_dev;
> +	} else {
> +		vbq->dev = dev;
> +	}
> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> +	if (output)
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> +	else
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> +	/* No minimum buffers limitation */
> +	vbq->min_buffers_needed = 0;
> +	vbq->drv_priv = cam;
> +	vbq->lock = &node->vdev_lock;
> +	vbq->supports_requests = true;
> +	vbq->requires_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_media_clean;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 dev_driver_string(dev), node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->vdev_lock;
> +	vdev->v4l2_dev = &cam->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> +	vdev->entity.ops = NULL;
> +	video_set_drvdata(vdev, cam);
> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->vdev_lock);
> +	INIT_LIST_HEAD(&node->buf_list);
> +	spin_lock_init(&node->buf_list_lock);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vb2_rel;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam->subdev.entity,
> +					    node->id, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam->subdev.entity,
> +					    node->id, &vdev->entity, 0,
> +					    link_flags);
> +	}
> +	if (ret)
> +		goto fail_vdev_ureg;
> +
> +	return 0;
> +
> +fail_vdev_ureg:
> +	video_unregister_device(vdev);
> +fail_vb2_rel:
> +	mutex_destroy(&node->vdev_lock);
> +	vb2_queue_release(vbq);
> +fail_media_clean:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static void
> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> +{
> +	video_unregister_device(&node->vdev);
> +	vb2_queue_release(&node->vbq);
> +	media_entity_cleanup(&node->vdev.entity);
> +	mutex_destroy(&node->vdev_lock);
> +}
> +
> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	/* Set up media device & pads */
> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> +	if (ret)
> +		return ret;
> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> +
> +	/* Set up v4l2 device */
> +	cam->v4l2_dev.mdev = &cam->media_dev;
> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> +		 "%s", dev_driver_string(dev));
> +	v4l2_set_subdevdata(&cam->subdev, cam);
> +
> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> +		goto fail_clean_media_entiy;
> +	}
> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> +
> +		node->id = node->desc.id;
> +		ret = mtk_cam_video_register_device(cam, node);
> +		if (ret)
> +			goto fail_vdev_unreg;
> +	}
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_vdev_unreg:
> +	for (i--; i >= 0; i--)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +fail_clean_media_entiy:
> +	media_entity_cleanup(&cam->subdev.entity);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +
> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> +	v4l2_device_unregister_subdev(&cam->subdev);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +	media_entity_cleanup(&cam->subdev.entity);
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> +		return -ENODEV;
> +	}
> +
> +	cam->seninf = sd;
> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam->seninf = NULL;
> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "No seninf subdev\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> +				    MEDIA_LNK_FL_IMMUTABLE |
> +				    MEDIA_LNK_FL_ENABLED);
> +	if (ret) {
> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> +			cam->seninf->entity.name, cam->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	v4l2_async_notifier_init(&cam->notifier);
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> +	if (ret) {
> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> +		return ret;
> +	}
> +
> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> +	if (ret) {
> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> +{
> +	v4l2_async_notifier_unregister(&cam->notifier);
> +	v4l2_async_notifier_cleanup(&cam->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 512 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 1200 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 640 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 288 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	/* This is a default image format */
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> +{
> +	unsigned int node_idx;
> +	int i;
> +
> +	node_idx = 0;
> +	/* Setup the output queue */
> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam)
> +{
> +	int ret;
> +
> +	cam->dev = &pdev->dev;
> +	mtk_cam_dev_queue_setup(cam);
> +
> +	spin_lock_init(&cam->pending_job_lock);
> +	spin_lock_init(&cam->running_job_lock);
> +	INIT_LIST_HEAD(&cam->pending_job_list);
> +	INIT_LIST_HEAD(&cam->running_job_list);
> +	mutex_init(&cam->op_lock);
> +
> +	/* v4l2 sub-device registration */
> +	ret = mtk_cam_v4l2_register(cam);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam);
> +	if (ret)
> +		goto fail_v4l2_unreg;
> +
> +	return 0;
> +
> +fail_v4l2_unreg:
> +	mutex_destroy(&cam->op_lock);
> +	mtk_cam_v4l2_unregister(cam);
> +
> +	return ret;
> +}
> +
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> +{
> +	mtk_cam_v4l2_async_unregister(cam);
> +	mtk_cam_v4l2_unregister(cam);
> +	mutex_destroy(&cam->op_lock);
> +}
> +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..0a340a1e65ea
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,244 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_H__
> +#define __MTK_CAM_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "mtk_cam-ipi.h"
> +
> +#define IMG_MAX_WIDTH		5376
> +#define IMG_MAX_HEIGHT		4032
> +#define IMG_MIN_WIDTH		80
> +#define IMG_MIN_HEIGHT		60
> +
> +/*
> + * ID enum value for struct mtk_cam_dev_node_desc:id
> + * or mtk_cam_video_device:id
> + */
> +enum  {
> +	MTK_CAM_P1_META_IN_0 = 0,
> +	MTK_CAM_P1_MAIN_STREAM_OUT,
> +	MTK_CAM_P1_PACKED_BIN_OUT,
> +	MTK_CAM_P1_META_OUT_0,
> +	MTK_CAM_P1_META_OUT_1,
> +	MTK_CAM_P1_META_OUT_2,
> +	MTK_CAM_P1_META_OUT_3,
> +	MTK_CAM_P1_TOTAL_NODES
> +};
> +
> +/* Supported image format list */
> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> +
> +/* Supported bayer pixel order */
> +#define MTK_CAM_RAW_PXL_ID_B		0
> +#define MTK_CAM_RAW_PXL_ID_GB		1
> +#define MTK_CAM_RAW_PXL_ID_GR		2
> +#define MTK_CAM_RAW_PXL_ID_R		3
> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> +
> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +/*
> + * struct mtk_cam_dev_request - MTK camera device request.
> + *
> + * @req: Embedded struct media request.
> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> + * @frame_work: work queue entry for frame transmission to SCP.
> + * @list: List entry of the object for @struct mtk_cam_dev:
> + *        pending_job_list or running_job_list.
> + * @timestamp: Start of frame timestamp in ns
> + *
> + */
> +struct mtk_cam_dev_request {
> +	struct media_request req;
> +	struct mtk_p1_frame_param frame_params;
> +	struct work_struct frame_work;
> +	struct list_head list;
> +	u64 timestamp;
> +};
> +
> +/*
> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> + *
> + * @vbb: Embedded struct vb2_v4l2_buffer.
> + * @list: List entry of the object for @struct mtk_cam_video_device:
> + *        buf_list.
> + * @daddr: The DMA address of this buffer.
> + * @scp_addr: The SCP address of this buffer which
> + *            is only supported for meta input node.
> + * @node_id: The vidoe node id which this buffer belongs to.
> + *
> + */
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer vbb;
> +	struct list_head list;
> +	/* Intenal part */
> +	dma_addr_t daddr;
> +	dma_addr_t scp_addr;
> +	unsigned int node_id;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> + *
> + * @id: id of the node
> + * @name: name of the node
> + * @cap: supported V4L2 capabilities
> + * @buf_type: supported V4L2 buffer type
> + * @dma_port: the dma ports associated to the node
> + * @link_flags: default media link flags
> + * @smem_alloc: using the smem_dev as alloc device or not
> + * @image: true for image node, false for meta node
> + * @num_fmts: the number of supported node formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum VB2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts: supported format
> + * @frmsizes: supported V4L2 frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	const char *name;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure
> + *
> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> + * @enabled: Indicate the video device is enabled or not
> + * @desc: The node description of video device
> + * @vdev_fmt: The V4L2 format of video device
> + * @vdev_pad: The media pad graph object of video device
> + * @vbq: A videobuf queue of video device
> + * @vdev: The video device instance
> + * @vdev_lock: Serializes vb2 queue and video device operations
> + * @buf_list: List for enqueue buffers
> + * @buf_list_lock: Lock used to protect buffer list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_format vdev_fmt;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct video_device vdev;
> +	/* Serializes vb2 queue and video device operations */
> +	struct mutex vdev_lock;
> +	struct list_head buf_list;
> +	/* Lock used to protect buffer list */
> +	spinlock_t buf_list_lock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @dev: Pointer to device.
> + * @smem_pdev: Pointer to shared memory device.
> + * @pipeline: Media pipeline information.
> + * @media_dev: Media device instance.
> + * @subdev: The V4L2 sub-device instance.
> + * @v4l2_dev: The V4L2 device driver instance.
> + * @notifier: The v4l2_device notifier data.
> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> + * @seninf: Pointer to the seninf sub-device.
> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> + * @streaming: Indicate the overall streaming status is on or off.
> + * @enabled_dmas: The enabled dma port information when streaming on.
> + * @enabled_count: Number of enabled video nodes
> + * @stream_count: Number of streaming video nodes
> + * @running_job_count: Nunber of running jobs in the HW driver.
> + * @pending_job_list: List to keep the media requests before en-queue into
> + *                    HW driver.
> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> + * @running_job_list: List to keep the media requests after en-queue into
> + *                    HW driver.
> + * @running_job_lock: Protect the running_job_list data.
> + * @op_lock: Serializes driver's VB2 callback operations.
> + *
> + */
> +struct mtk_cam_dev {
> +	struct device *dev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int enabled_dmas;
> +	unsigned int enabled_count;
> +	unsigned int stream_count;
> +	unsigned int running_job_count;
> +	struct list_head pending_job_list;
> +	/* Protect the pending_job_list data */
> +	spinlock_t pending_job_lock;
> +	struct list_head running_job_list;
> +	/* Protect the running_job_list data & running_job_count */
> +	spinlock_t running_job_lock;
> +	/* Serializes driver's VB2 callback operations */
> +	struct mutex op_lock;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> +				   unsigned int frame_seq_no);
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				  unsigned int frame_seq_no);
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no);
> +
> +#endif /* __MTK_CAM_H__ */
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-04-02 16:45       ` Dafna Hirschfeld
  0 siblings, 0 replies; 388+ messages in thread
From: Dafna Hirschfeld @ 2020-04-02 16:45 UTC (permalink / raw)
  To: Jungo Lin, tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg,
	mchehab
  Cc: shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media



On 19.12.19 06:49, Jungo Lin wrote:
> This patch adds the Mediatek ISP P1 HW control device driver.
> It handles the ISP HW configuration, provides interrupt handling and
> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> implement standard V4L2 video driver that utilizes V4L2 and media
> framework APIs. It supports one media device, one sub-device and
> several video devices during initialization. Moreover, it also connects
> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> co-process via SCP communication to compose ISP registers in the
> firmware.
> 
> (The current metadata interface used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> ---
> Changes from v6:
>   - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>   - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>   - Correct auto suspend timer value for suspend/resume issue
>   - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>   - Fix KE due to no sen-inf sub-device
> ---
>   drivers/media/platform/mtk-isp/Kconfig        |   20 +
>   .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>   .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>   .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>   .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>   .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>   9 files changed, 3377 insertions(+)
>   create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> 
> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> new file mode 100644
> index 000000000000..f86e1b59ad1e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/Kconfig
> @@ -0,0 +1,20 @@
> +config VIDEO_MEDIATEK_ISP_PASS1
> +	tristate "Mediatek ISP Pass 1 driver"
> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> +	depends on ARCH_MEDIATEK
> +	select V4L2_FWNODE
> +	select VIDEOBUF2_VMALLOC
> +	select VIDEOBUF2_DMA_CONTIG
> +	select MTK_SCP
> +	default n
> +	help
> +		Pass 1 driver controls 3A (auto-focus, exposure,
> +		and white balance) with tuning feature and outputs
> +		the captured image buffers in Mediatek's camera system.
> +
> +		Choose Y if you want to use Mediatek SoCs to create image
> +		captured application such as video recording and still image
> +		capturing.
> +
> +		To compile this driver as a module, choose M here; the module
> +		will be called mtk-cam-isp.
> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> new file mode 100644
> index 000000000000..ce79d283b209
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> new file mode 100644
> index 000000000000..53b54d3c26a0
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +mtk-cam-isp-objs += mtk_cam.o
> +mtk-cam-isp-objs += mtk_cam-hw.o
> +
> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> \ No newline at end of file
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> new file mode 100644
> index 000000000000..4065d0d29b7f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> @@ -0,0 +1,636 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/atomic.h>
> +#include <linux/clk.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/module.h>
> +#include <linux/remoteproc/mtk_scp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/remoteproc.h>
> +#include <linux/sched.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +#include <linux/vmalloc.h>
> +
> +#include <media/v4l2-event.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +#include "mtk_cam-regs.h"
> +
> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> +#define MTK_ISP_CQ_BUFFER_COUNT			3
> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> +
> +/*
> + *
> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> + * In current driver, only supports CAM B.
> + *
> + */
> +#define MTK_ISP_CAM_ID_B			3
> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> +
> +static void isp_tx_frame_worker(struct work_struct *work)
> +{
> +	struct mtk_cam_dev_request *req =
> +		container_of(work, struct mtk_cam_dev_request, frame_work);
> +	struct mtk_cam_dev *cam =
> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> +
> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> +
> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> +		dev_err(dev, "wrong IPI len:%d\n", len);
> +		return;
> +	}
> +
> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> +		return;
> +
> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> +}
> +
> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	int ret;
> +
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI cmd\n");
> +		return ret;
> +	}
> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> +			       isp_composer_handler, p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register IPI frame\n");
> +		goto unreg_ipi_cmd;
> +	}
> +
> +	p1_dev->composer_wq =
> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> +					WQ_FREEZABLE);
> +	if (!p1_dev->composer_wq) {
> +		dev_err(dev, "failed to alloc composer workqueue\n");
> +		goto unreg_ipi_frame;
> +	}
> +
> +	return 0;
> +
> +unreg_ipi_frame:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +unreg_ipi_cmd:
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +
> +	return ret;
> +}
> +
> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	destroy_workqueue(p1_dev->composer_wq);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> +}
> +
> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> +
> +	/*
> +	 * Passed coherent reserved memory info. for SCP firmware usage.
> +	 * This buffer is used for SCP's ISP composer to compose.
> +	 * The size of is fixed to 0x200000 for the requirement of composer.
> +	 */
> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +
> +	isp_composer_uninit(p1_dev);
> +}
> +
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> +		       struct p1_config_param *config_param)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> +	memcpy(&composer_tx_cmd.config_param, config_param,
> +	       sizeof(*config_param));
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> +{
> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> +	composer_tx_cmd.is_stream_on = on;
> +
> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> +}
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = rproc_boot(p1_dev->rproc_handle);
> +	if (ret) {
> +		dev_err(dev, "failed to rproc_boot\n");
> +		return ret;
> +	}
> +
> +	ret = isp_composer_init(p1_dev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_get_sync(dev);
> +	isp_composer_hw_init(p1_dev);
> +
> +	p1_dev->enqueued_frame_seq_no = 0;
> +	p1_dev->dequeued_frame_seq_no = 0;
> +	p1_dev->composed_frame_seq_no = 0;
> +	p1_dev->sof_count = 0;
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	isp_composer_hw_deinit(p1_dev);
> +	pm_runtime_mark_last_busy(dev);
> +	pm_runtime_put_autosuspend(dev);
> +	rproc_shutdown(p1_dev->rproc_handle);
> +
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> +			 struct mtk_cam_dev_request *req)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> +
> +	/* Accumulated frame sequence number */
> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> +
> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> +	queue_work(p1_dev->composer_wq, &req->frame_work);
> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no,
> +		cam->running_job_count);
> +}
> +
> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> +			       unsigned int dequeued_frame_seq_no)
> +{
> +	dma_addr_t base_addr = p1_dev->composer_iova;
> +	struct device *dev = p1_dev->dev;
> +	struct mtk_cam_dev_request *req;
> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> +	unsigned int addr_offset;
> +
> +	/* Send V4L2_EVENT_FRAME_SYNC event */
> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +
> +	p1_dev->sof_count += 1;
> +	/* Save frame information */
> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> +
> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> +	if (req)
> +		req->timestamp = ktime_get_boottime_ns();
> +
> +	/* Update CQ base address if needed */
> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> +		dev_dbg(dev,
> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> +			composed_frame_seq_no, dequeued_frame_seq_no);
> +		return;
> +	}
> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> +	dev_dbg(dev,
> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> +}
> +
> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> +{
> +	u32 val;
> +
> +	dev_err(p1_dev->dev,
> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> +	dev_err(p1_dev->dev,
> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> +
> +	/* Disable DMA error mask to avoid too much error log */
> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> +}
> +
> +static irqreturn_t isp_irq_cam(int irq, void *data)
> +{
> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> +	struct device *dev = p1_dev->dev;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int irq_status, err_status, dma_status;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> +
> +	/*
> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> +	 * If these two ISRs come together, print warning msg to hint.
> +	 */
> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> +
> +	/* De-queue frame */
> +	if (irq_status & SW_PASS1_DON_ST) {
> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> +					      p1_dev->dequeued_frame_seq_no);
> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> +	}
> +
> +	/* Save frame info. & update CQ address for frame HW en-queue */
> +	if (irq_status & SOF_INT_ST)
> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> +
> +	/* Check ISP error status */
> +	if (err_status) {
> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> +		/* Show DMA errors in detail */
> +		if (err_status & DMA_ERR_ST)
> +			isp_irq_handle_dma_err(p1_dev);
> +	}
> +
> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> +		p1_dev->sof_count, irq_status, dma_status,
> +		dequeued_frame_seq_no);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> +			       struct platform_device *pdev)
> +{
> +	struct device *dev = p1_dev->dev;
> +	dma_addr_t addr;
> +	void *ptr;
> +	int ret;
> +
> +	p1_dev->scp = scp_get(pdev);
> +	if (!p1_dev->scp) {
> +		dev_err(dev, "failed to get scp device\n");
> +		return -ENODEV;
> +	}
> +
> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> +
> +	/*
> +	 * Allocate coherent reserved memory for SCP firmware usage.
> +	 * The size of SCP composer's memory is fixed to 0x200000
> +	 * for the requirement of firmware.
> +	 */
> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> +	if (!ptr) {
> +		ret = -ENOMEM;
> +		goto fail_put_scp;
> +	}
> +
> +	p1_dev->composer_scp_addr = addr;
> +	p1_dev->composer_virt_addr = ptr;
> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> +
> +	/*
> +	 * This reserved memory is also be used by ISP P1 HW.
> +	 * Need to get iova address for ISP P1 DMA.
> +	 */
> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map scp iova\n");
> +		ret = -ENOMEM;
> +		goto fail_free_mem;
> +	}
> +	p1_dev->composer_iova = addr;
> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> +
> +	return 0;
> +
> +fail_free_mem:
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +fail_put_scp:
> +	scp_put(p1_dev->scp);
> +
> +	return ret;
> +}
> +
> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> +{
> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> +			  p1_dev->composer_virt_addr,
> +			  p1_dev->composer_scp_addr);
> +	p1_dev->composer_scp_addr = 0;
> +	scp_put(p1_dev->scp);
> +}
> +
> +static int mtk_isp_pm_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Disable ISP's view finder and wait for TG idle if possible */
> +	dev_dbg(dev, "cam suspend, disable VF\n");
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> +
> +	/* Disable CMOS */
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Force ISP HW to idle */
> +	ret = pm_runtime_force_suspend(dev);
> +	if (ret) {
> +		dev_err(dev, "failed to force suspend:%d\n", ret);
> +		goto reenable_hw;
> +	}
> +
> +	return 0;
> +
> +reenable_hw:
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return ret;
> +}
> +
> +static int mtk_isp_pm_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	u32 val;
> +	int ret;
> +
> +	dev_dbg(dev, "- %s\n", __func__);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	/* Force ISP HW to resume */
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret)
> +		return ret;
> +
> +	/* Enable CMOS */
> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> +
> +	/* Enable VF */
> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_suspend(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	dev_dbg(dev, "%s:disable clock\n", __func__);
> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_runtime_resume(struct device *dev)
> +{
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +	int ret;
> +
> +	dev_dbg(dev, "%s:enable clock\n", __func__);
> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to enable clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_probe(struct platform_device *pdev)
> +{
> +	/* List of clocks required by isp cam */
> +	static const char * const clk_names[] = {
> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> +	};
> +	struct mtk_isp_p1_device *p1_dev;
> +	struct device *dev = &pdev->dev;
> +	struct resource *res;
> +	int irq, ret, i;
> +
> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> +	if (!p1_dev)
> +		return -ENOMEM;
> +
> +	p1_dev->dev = dev;
> +	dev_set_drvdata(dev, p1_dev);
> +
> +	/*
> +	 * Now only support single CAM with CAM B.
> +	 * Get CAM B register base with CAM B index.
> +	 * Support multiple CAMs in future.
> +	 */
> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> +	p1_dev->regs = devm_ioremap_resource(dev, res);
> +	if (IS_ERR(p1_dev->regs)) {
> +		dev_err(dev, "failed to map reister base\n");
> +		return PTR_ERR(p1_dev->regs);
> +	}
> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> +
> +	/*
> +	 * The cam_sys unit only supports reg., but has no IRQ support.
> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> +	 */
> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> +	if (!irq) {
> +		dev_err(dev, "failed to get irq\n");
> +		return -ENODEV;
> +	}
> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> +			       p1_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to request irq=%d\n", irq);
> +		return ret;
> +	}
> +	dev_dbg(dev, "registered irq=%d\n", irq);
> +	spin_lock_init(&p1_dev->spinlock_irq);
> +
> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> +	if (!p1_dev->clks)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < p1_dev->num_clks; ++i)
> +		p1_dev->clks[i].id = clk_names[i];
> +
> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> +	if (ret) {
> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> +	if (ret)
> +		return ret;
> +
> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> +	pm_runtime_use_autosuspend(dev);
> +	pm_runtime_enable(dev);
> +
> +	/* Initialize the v4l2 common part */
> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> +	if (ret) {
> +		isp_teardown_scp_rproc(p1_dev);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int mtk_isp_remove(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> +
> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> +	pm_runtime_dont_use_autosuspend(dev);
> +	pm_runtime_disable(dev);
> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +	isp_teardown_scp_rproc(p1_dev);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops mtk_isp_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> +			   NULL)
> +};
> +
> +static const struct of_device_id mtk_isp_of_ids[] = {
> +	{.compatible = "mediatek,mt8183-camisp",},
> +	{}
> +};
> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> +
> +static struct platform_driver mtk_isp_driver = {
> +	.probe   = mtk_isp_probe,
> +	.remove  = mtk_isp_remove,
> +	.driver  = {
> +		.name  = "mtk-cam-p1",
> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> +		.pm     = &mtk_isp_pm_ops,
> +	}
> +};
> +
> +module_platform_driver(mtk_isp_driver);
> +
> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> new file mode 100644
> index 000000000000..837662f92a5e
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> @@ -0,0 +1,64 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_HW_H__
> +#define __MTK_CAM_HW_H__
> +
> +#include <linux/types.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ipi.h"
> +
> +/*
> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> + *
> + * @dev: Pointer to device.
> + * @scp_pdev: Pointer to SCP platform device.
> + * @rproc_handle: Pointer to new remoteproc instance.
> + * @cam_dev: Embedded struct cam_dev
> + * @regs: Camera ISP HW base register address
> + * @num_clks: The number of driver's clocks
> + * @clks: The clock data array
> + * @spinlock_irq: Used to protect register read/write data
> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> + * @composed_frame_seq_no: Frame sequence number of composed frame
> + * @timestamp: Frame timestamp in ns
> + * @sof_count: SOF counter
> + * @composer_wq: The work queue for frame request composing
> + * @composer_scp_addr: SCP address of ISP composer memory
> + * @composer_iova: DMA address of ISP composer memory
> + * @virt_addr: Virtual address of ISP composer memory
> + *
> + */
> +struct mtk_isp_p1_device {
> +	struct device *dev;
> +	struct mtk_scp *scp;
> +	struct rproc *rproc_handle;
> +	struct mtk_cam_dev cam_dev;
> +	void __iomem *regs;
> +	unsigned int num_clks;
> +	struct clk_bulk_data *clks;
> +	/* Used to protect register read/write data */
> +	spinlock_t spinlock_irq;
> +	unsigned int enqueued_frame_seq_no;
> +	unsigned int dequeued_frame_seq_no;
> +	unsigned int composed_frame_seq_no;
> +	u8 sof_count;
> +	struct workqueue_struct *composer_wq;
> +	dma_addr_t composer_scp_addr;
> +	dma_addr_t composer_iova;
> +	void *composer_virt_addr;
> +};
> +
> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> +		       struct p1_config_param *config_param);
> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> +			 struct mtk_cam_dev_request *req);
> +
> +#endif /* __MTK_CAM_HW_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> new file mode 100644
> index 000000000000..981b634dd91f
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> @@ -0,0 +1,222 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_IPI_H__
> +#define __MTK_CAM_IPI_H__
> +
> +#include <linux/types.h>
> +
> +/*
> + * struct img_size - Image size information.
> + *
> + * @w: Image width, the unit is pixel
> + * @h: Image height, the unit is pixel
> + * @xsize: Bytes per line based on width.
> + * @stride: Bytes per line when changing line.
> + *          Stride is based on xsize + HW constrain(byte align).
> + *
> + */
> +struct img_size {
> +	u32 w;
> +	u32 h;
> +	u32 xsize;
> +	u32 stride;
> +} __packed;
> +
> +/*
> + * struct p1_img_crop - image corp information
> + *
> + * @left: The left of crop area.
> + * @top: The top of crop area.
> + * @width: The width of crop area.
> + * @height: The height of crop area.
> + *
> + */
> +struct p1_img_crop {
> +	u32 left;
> +	u32 top;
> +	u32 width;
> +	u32 height;
> +} __packed;
> +
> +/*
> + * struct dma_buffer - DMA buffer address information
> + *
> + * @iova: DMA address for ISP DMA device
> + * @scp_addr: SCP address for external co-process unit
> + *
> + */
> +struct dma_buffer {
> +	u32 iova;
> +	u32 scp_addr;
> +} __packed;
> +
> +/*
> + * struct p1_img_output - ISP P1 image output information
> + *
> + * @buffer: DMA buffer address of image.
> + * @size: The image size configuration.
> + * @crop: The crop configuration.
> + * @pixel_bits: The bits per image pixel.
> + * @img_fmt: The image format.
> + *
> + */
> +struct p1_img_output {
> +	struct dma_buffer buffer;
> +	struct img_size size;
> +	struct p1_img_crop crop;
> +	u8 pixel_bits;
> +	u32 img_fmt;
> +} __packed;
> +
> +/*
> + * struct cfg_in_param - Image input parameters structure.
> + *                       Normally, it comes from sensor information.
> + *
> + * @continuous: Indicate the sensor mode. Continuous or single shot.
> + * @subsample: Indicate to enables SOF subsample or not.
> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> + * @data_pattern: Describe input data pattern.
> + * @raw_pixel_id: Bayer sequence.
> + * @tg_fps: The fps rate of TG (time generator).
> + * @img_fmt: The image format of input source.
> + * @p1_img_crop: The crop configuration of input source.
> + *
> + */
> +struct cfg_in_param {
> +	u8 continuous;
> +	u8 subsample;
> +	u8 pixel_mode;
> +	u8 data_pattern;
> +	u8 raw_pixel_id;
> +	u16 tg_fps;
> +	u32 img_fmt;
> +	struct p1_img_crop crop;
> +} __packed;
> +
> +/*
> + * struct cfg_main_out_param - The image output parameters of main stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @pure_raw: Indicate the image path control.
> + *            True: pure raw
> + *            False: processing raw
> + * @pure_raw_pack: Indicate the image is packed or not.
> + *                 True: packed mode
> + *                 False: unpacked mode
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_main_out_param {
> +	u8 bypass;
> +	u8 pure_raw;
> +	u8 pure_raw_pack;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct cfg_resize_out_param - The image output parameters of
> + *                               packed out stream.
> + *
> + * @bypass: Indicate this device is enabled or disabled or not.
> + * @p1_img_output: The output image information.
> + *
> + */
> +struct cfg_resize_out_param {
> +	u8 bypass;
> +	struct p1_img_output output;
> +} __packed;
> +
> +/*
> + * struct p1_config_param - ISP P1 configuration parameters.
> + *
> + * @cfg_in_param: The Image input parameters.
> + * @cfg_main_param: The main output image parameters.
> + * @cfg_resize_out_param: The packed output image parameters.
> + * @enabled_dmas: The enabled DMA port information.
> + *
> + */
> +struct p1_config_param {
> +	struct cfg_in_param cfg_in_param;
> +	struct cfg_main_out_param cfg_main_param;
> +	struct cfg_resize_out_param cfg_resize_param;
> +	u32 enabled_dmas;
> +} __packed;
> +
> +/*
> + * struct P1_meta_frame - ISP P1 meta frame information.
> + *
> + * @enabled_dma: The enabled DMA port information.
> + * @vb_index: The VB2 index of meta buffer.
> + * @meta_addr: DMA buffer address of meta buffer.
> + *
> + */
> +struct P1_meta_frame {
> +	u32 enabled_dma;
> +	u32 vb_index;
> +	struct dma_buffer meta_addr;
> +} __packed;
> +
> +/*
> + * struct isp_init_info - ISP P1 composer init information.
> + *
> + * @hw_module: The ISP Camera HW module ID.
> + * @cq_addr: The DMA address of composer memory.
> + *
> + */
> +struct isp_init_info {
> +	u8 hw_module;
> +	struct dma_buffer cq_addr;
> +} __packed;
> +
> +/*
> + * struct isp_ack_info - ISP P1 IPI command ack information.
> + *
> + * @cmd_id: The IPI command ID is acked.
> + * @frame_seq_no: The IPI frame sequence number is acked.
> + *
> + */
> +struct isp_ack_info {
> +	u8 cmd_id;
> +	u32 frame_seq_no;
> +} __packed;
> +
> +/*
> + * The IPI command enumeration.
> + */
> +enum mtk_isp_scp_cmds {
> +	ISP_CMD_INIT,
> +	ISP_CMD_CONFIG,
> +	ISP_CMD_STREAM,
> +	ISP_CMD_DEINIT,
> +	ISP_CMD_ACK,
> +	ISP_CMD_FRAME_ACK,
> +	ISP_CMD_RESERVED,
> +};
> +
> +/*
> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> + *
> + * @cmd_id: The IPI command ID.
> + * @init_param: The init formation for ISP_CMD_INIT.
> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> + *
> + */
> +struct mtk_isp_scp_p1_cmd {
> +	u8 cmd_id;
> +	union {
> +		struct isp_init_info init_param;
> +		struct p1_config_param config_param;
> +		u32 enabled_dmas;
> +		struct P1_meta_frame meta_frame;
> +		u8 is_stream_on;
> +		struct isp_ack_info ack_info;
> +	};
> +} __packed;
> +
> +#endif /* __MTK_CAM_IPI_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> new file mode 100644
> index 000000000000..ab2277f45fa4
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> @@ -0,0 +1,95 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_REGS_H__
> +#define __MTK_CAM_REGS_H__
> +
> +/* ISP interrupt enable */
> +#define REG_CTL_RAW_INT_EN		0x0020
> +#define DMA_ERR_INT_EN			BIT(29)
> +
> +/* ISP interrupt status */
> +#define REG_CTL_RAW_INT_STAT		0x0024
> +#define VS_INT_ST			BIT(0)
> +#define TG_ERR_ST			BIT(4)
> +#define TG_GBERR_ST			BIT(5)
> +#define CQ_CODE_ERR_ST			BIT(6)
> +#define CQ_APB_ERR_ST			BIT(7)
> +#define CQ_VS_ERR_ST			BIT(8)
> +#define HW_PASS1_DON_ST			BIT(11)
> +#define SOF_INT_ST			BIT(12)
> +#define AMX_ERR_ST			BIT(15)
> +#define RMX_ERR_ST			BIT(16)
> +#define BMX_ERR_ST			BIT(17)
> +#define RRZO_ERR_ST			BIT(18)
> +#define AFO_ERR_ST			BIT(19)
> +#define IMGO_ERR_ST			BIT(20)
> +#define AAO_ERR_ST			BIT(21)
> +#define PSO_ERR_ST			BIT(22)
> +#define LCSO_ERR_ST			BIT(23)
> +#define BNR_ERR_ST			BIT(24)
> +#define LSCI_ERR_ST			BIT(25)
> +#define DMA_ERR_ST			BIT(29)
> +#define SW_PASS1_DON_ST			BIT(30)
> +
> +/* ISP interrupt 2 status */
> +#define REG_CTL_RAW_INT2_STAT		0x0034
> +#define AFO_DONE_ST			BIT(5)
> +#define AAO_DONE_ST			BIT(7)
> +
> +/* Configures sensor mode */
> +#define REG_TG_SEN_MODE			0x0230
> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_VF_CON			0x0234
> +#define TG_VF_CON_VFDATA_EN		BIT(0)
> +
> +/* View finder mode control */
> +#define REG_TG_INTER_ST			0x026c
> +#define TG_CS_MASK			0x3f00
> +#define TG_IDLE_ST			BIT(8)
> +
> +/* IMGO error status register */
> +#define REG_IMGO_ERR_STAT		0x1360
> +/* RRZO error status register */
> +#define REG_RRZO_ERR_STAT		0x1364
> +/* AAO error status register */
> +#define REG_AAO_ERR_STAT		0x1368
> +/* AFO error status register */
> +#define REG_AFO_ERR_STAT		0x136c
> +/* LCSO error status register */
> +#define REG_LCSO_ERR_STAT		0x1370
> +/* BPCI error status register */
> +#define REG_BPCI_ERR_STAT		0x137c
> +/* LSCI error status register */
> +#define REG_LSCI_ERR_STAT		0x1384
> +/* LMVO error status register */
> +#define REG_LMVO_ERR_STAT		0x1390
> +/* FLKO error status register */
> +#define REG_FLKO_ERR_STAT		0x1394
> +/* PSO error status register */
> +#define REG_PSO_ERR_STAT		0x13a0
> +
> +/* CQ0 base address */
> +#define REG_CQ_THR0_BASEADDR		0x0198
> +/* Frame sequence number */
> +#define REG_FRAME_SEQ_NUM		0x13b8
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR		( \
> +					TG_ERR_ST |\
> +					TG_GBERR_ST |\
> +					CQ_CODE_ERR_ST |\
> +					CQ_APB_ERR_ST |\
> +					CQ_VS_ERR_ST |\
> +					BNR_ERR_ST |\
> +					RMX_ERR_ST |\
> +					BMX_ERR_ST |\
> +					BNR_ERR_ST |\
> +					LSCI_ERR_ST |\
> +					DMA_ERR_ST)
> +
> +#endif	/* __MTK_CAM_REGS_H__ */
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> new file mode 100644
> index 000000000000..23fdb8b4abc5
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> @@ -0,0 +1,2087 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2019 MediaTek Inc.
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <linux/of_platform.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-hw.h"
> +
> +#define R_IMGO		BIT(0)
> +#define R_RRZO		BIT(1)
> +#define R_AAO		BIT(3)
> +#define R_AFO		BIT(4)
> +#define R_LCSO		BIT(5)
> +#define R_LMVO		BIT(7)
> +#define R_FLKO		BIT(8)
> +#define R_PSO		BIT(10)
> +
> +#define MTK_ISP_ONE_PIXEL_MODE		1
> +#define MTK_ISP_MIN_RESIZE_RATIO	6
> +#define MTK_ISP_MAX_RUNNING_JOBS	3
> +
> +#define MTK_CAM_CIO_PAD_SRC		4
> +#define MTK_CAM_CIO_PAD_SINK		11
> +
> +static inline struct mtk_cam_video_device *
> +file_to_mtk_cam_node(struct file *__file)
> +{
> +	return container_of(video_devdata(__file),
> +		struct mtk_cam_video_device, vdev);
> +}
> +
> +static inline struct mtk_cam_video_device *
> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> +{
> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> +}
> +
> +static inline struct mtk_cam_dev_request *
> +mtk_cam_req_to_dev_req(struct media_request *__req)
> +{
> +	return container_of(__req, struct mtk_cam_dev_request, req);
> +}
> +
> +static inline struct mtk_cam_dev_buffer *
> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> +{
> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> +}
> +
> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> +				 struct mtk_cam_dev_request *req,
> +				 enum vb2_buffer_state state)
> +{
> +	struct media_request_object *obj, *obj_prev;
> +	unsigned long flags;
> +	u64 ts_eof = ktime_get_boottime_ns();
> +
> +	if (!cam->streaming)
> +		return;
> +
> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> +
> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> +		struct vb2_buffer *vb;
> +		struct mtk_cam_dev_buffer *buf;
> +		struct mtk_cam_video_device *node;
> +
> +		if (!vb2_request_object_is_buffer(obj))
> +			continue;
> +		vb = container_of(obj, struct vb2_buffer, req_obj);
> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +		spin_lock_irqsave(&node->buf_list_lock, flags);
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> +			vb->timestamp = ts_eof;
> +		else
> +			vb->timestamp = req->timestamp;
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +}
> +
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +			return req;
> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +
> +	return NULL;
> +}
> +
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> +				   unsigned int frame_seq_no)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> +			req->frame_params.frame_seq_no, frame_seq_no);
> +
> +		/* Match by the en-queued request number */
> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> +			list_del(&req->list);
> +			break;
> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> +			cam->running_job_count--;
> +			/* Pass to user space for frame drop */
> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
Hi, I see that frame_params.frame_seq_no is incremented when a request is queued
and frame_seq_no is read from a register, so if the first is lower than the latter
it means userspace was queueing request too slowly and missed frames right?
So userspace will have to catch up in order not to get "ERROR" buffers.
Maybe the driver can just skip lost frames. In the rkisp1 for example,
if there is no buffer available, the driver just write to a dummy buffer that is
never sent to userspace. This way userspace always get valid buffers back when
dequeueing.

> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> +				 req->frame_params.frame_seq_no);
> +			list_del(&req->list);
> +		} else {
> +			break;
Does this case can ever occur?

Thanks,
Dafna

> +		}
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> +		list_del(&req->list);
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +}
> +
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> +{
> +	struct mtk_cam_dev_request *req, *req_prev;
> +	unsigned long flags;
> +
> +	if (!cam->streaming) {
> +		dev_dbg(cam->dev, "stream is off\n");
> +		return;
> +	}
> +
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	spin_lock_irqsave(&cam->running_job_lock, flags);
> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> +			dev_dbg(cam->dev, "jobs are full\n");
> +			break;
> +		}
> +		cam->running_job_count++;
> +		list_del(&req->list);
> +		list_add_tail(&req->list, &cam->running_job_list);
> +		mtk_isp_req_enqueue(cam, req);
> +	}
> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +}
> +
> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req;
> +
> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> +
> +	return &cam_dev_req->req;
> +}
> +
> +static void mtk_cam_req_free(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> +
> +	kfree(cam_dev_req);
> +}
> +
> +static void mtk_cam_req_queue(struct media_request *req)
> +{
> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> +					       media_dev);
> +	unsigned long flags;
> +
> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> +	vb2_request_queue(req);
> +
> +	/* add to pending job list */
> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> +
> +	mtk_cam_dev_req_try_queue(cam);
> +}
> +
> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> +{
> +	switch (pix_fmt) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return 8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return 10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return 12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return 14;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> +			     struct v4l2_pix_format_mplane *mp)
> +{
> +	unsigned int bpl, ppl;
> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> +	unsigned int width = mp->width;
> +
> +	bpl = 0;
> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> +		/* Bayer encoding format & 2 bytes alignment */
> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> +		/*
> +		 * The FULL-G encoding format
> +		 * 1 G component per pixel
> +		 * 1 R component per 4 pixel
> +		 * 1 B component per 4 pixel
> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> +		 */
> +		ppl = DIV_ROUND_UP(width * 6, 4);
> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> +
> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> +		if (pixel_bits == 10)
> +			bpl = ALIGN(bpl, 4);
> +		else
> +			bpl = ALIGN(bpl, 8);
> +	}
> +	/*
> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> +	 */
> +	bpl = ALIGN(bpl, 4);
> +
> +	mp->plane_fmt[0].bytesperline = bpl;
> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> +
> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> +}
> +
> +static const struct v4l2_format *
> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> +{
> +	int i;
> +	const struct v4l2_format *dev_fmt;
> +
> +	for (i = 0; i < desc->num_fmts; i++) {
> +		dev_fmt = &desc->fmts[i];
> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> +			return dev_fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +/* Get the default format setting */
> +static void
> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> +			     struct mtk_cam_dev_node_desc *queue_desc,
> +			     struct v4l2_format *dest)
> +{
> +	const struct v4l2_format *default_fmt =
> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> +
> +	dest->type = queue_desc->buf_type;
> +
> +	/* Configure default format based on node type */
> +	if (!queue_desc->image) {
> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> +		return;
> +	}
> +
> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> +	dest->fmt.pix_mp.num_planes = 1;
> +
> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +}
> +
> +/* Utility functions */
> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_B;
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GB;
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_GR;
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_RAW_PXL_ID_R;
> +	default:
> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_sensor_fmt(unsigned int fmt)
> +{
> +	switch (fmt) {
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static unsigned int get_img_fmt(unsigned int fourcc)
> +{
> +	switch (fourcc) {
> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> +		return MTK_CAM_IMG_FMT_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> +		return MTK_CAM_IMG_FMT_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> +		return MTK_CAM_IMG_FMT_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> +		return MTK_CAM_IMG_FMT_BAYER14;
> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> +	default:
> +		return MTK_CAM_IMG_FMT_UNKNOWN;
> +	}
> +}
> +
> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> +			  struct p1_img_output *out_fmt, int sd_width,
> +			  int sd_height)
> +{
> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> +
> +	/* Check output & input image size dimension */
> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> +			node_id);
> +		return -EINVAL;
> +	}
> +
> +	/* Check resize ratio for resize out stream due to HW constraint */
> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> +		return -EINVAL;
> +	}
> +
> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    !out_fmt->pixel_bits) {
> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> +		return -EINVAL;
> +	}
> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> +
> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> +
> +	out_fmt->crop.left = 0;
> +	out_fmt->crop.top = 0;
> +	out_fmt->crop.width = sd_width;
> +	out_fmt->crop.height = sd_height;
> +
> +	dev_dbg(cam->dev,
> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> +		node_id, out_fmt->size.w, out_fmt->size.h,
> +		out_fmt->size.stride, out_fmt->size.xsize,
> +		out_fmt->crop.width, out_fmt->crop.height);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	cam->enabled_count = 0;
> +	cam->enabled_dmas = 0;
> +	cam->stream_count = 0;
> +	cam->running_job_count = 0;
> +
> +	/* Get the enabled meta DMA ports */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		if (!cam->vdev_nodes[i].enabled)
> +			continue;
> +		cam->enabled_count++;
> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> +	}
> +
> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> +		cam->enabled_dmas);
> +}
> +
> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	struct p1_config_param config_param;
> +	struct cfg_in_param *cfg_in_param;
> +	struct v4l2_subdev_format sd_fmt;
> +	int sd_width, sd_height, sd_code;
> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> +	int ret;
> +
> +	/* Get sensor format configuration */
> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> +	if (ret) {
> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> +		return ret;
> +	}
> +	sd_width = sd_fmt.format.width;
> +	sd_height = sd_fmt.format.height;
> +	sd_code = sd_fmt.format.code;
> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> +		sd_code);
> +
> +	memset(&config_param, 0, sizeof(config_param));
> +
> +	/* Update cfg_in_param */
> +	cfg_in_param = &config_param.cfg_in_param;
> +	cfg_in_param->continuous = true;
> +	/* Fix to one pixel mode in default */
> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> +	cfg_in_param->crop.width = sd_width;
> +	cfg_in_param->crop.height = sd_height;
> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> +		return -EINVAL;
> +	}
> +
> +	/* Update cfg_main_param */
> +	config_param.cfg_main_param.pure_raw = true;
> +	config_param.cfg_main_param.pure_raw_pack = true;
> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> +			     &config_param.cfg_main_param.output,
> +			     sd_width, sd_height);
> +	if (ret)
> +		return ret;
> +
> +	/* Update cfg_resize_param */
> +	if (enabled_dma_ports & R_RRZO) {
> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> +				     &config_param.cfg_resize_param.output,
> +				     sd_width, sd_height);
> +		if (ret)
> +			return ret;
> +	} else {
> +		config_param.cfg_resize_param.bypass = true;
> +	}
> +
> +	/* Update enabled_dmas */
> +	config_param.enabled_dmas = enabled_dma_ports;
> +	mtk_isp_hw_config(cam, &config_param);
> +	dev_dbg(dev, "%s done\n", __func__);
> +
> +	return 0;
> +}
> +
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> +				  unsigned int frame_seq_no)
> +{
> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = frame_seq_no,
> +	};
> +
> +	v4l2_event_queue(cam->subdev.devnode, &event);
> +}
> +
> +static struct v4l2_subdev *
> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> +{
> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> +	struct device *dev = cam->dev;
> +	struct media_entity *entity;
> +	struct v4l2_subdev *sensor;
> +
> +	sensor = NULL;
> +	media_device_for_each_entity(entity, mdev) {
> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> +			entity->name, entity->function, entity->stream_count);
> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> +		    entity->stream_count) {
> +			sensor = media_entity_to_v4l2_subdev(entity);
> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> +			break;
> +		}
> +	}
> +
> +	if (!sensor)
> +		dev_err(dev, "no seninf connected\n");
> +
> +	return sensor;
> +}
> +
> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "no seninf connected\n");
> +		return -ENODEV;
> +	}
> +
> +	/* Get active sensor from graph topology */
> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> +	if (!cam->sensor)
> +		return -ENODEV;
> +
> +	/* Seninf must stream on first */
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> +	if (ret) {
> +		dev_err(dev, "failed to stream on %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		goto fail_seninf_off;
> +	}
> +
> +	ret = mtk_cam_dev_isp_config(cam);
> +	if (ret)
> +		goto fail_sensor_off;
> +
> +	cam->streaming = true;
> +	mtk_isp_stream(cam, 1);
> +	mtk_cam_dev_req_try_queue(cam);
> +	dev_dbg(dev, "streamed on Pass 1\n");
> +
> +	return 0;
> +
> +fail_sensor_off:
> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +fail_seninf_off:
> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->sensor->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> +	if (ret) {
> +		dev_err(dev, "failed to stream off %s:%d\n",
> +			cam->seninf->entity.name, ret);
> +		return -EPERM;
> +	}
> +
> +	cam->streaming = false;
> +	mtk_isp_stream(cam, 0);
> +	mtk_isp_hw_release(cam);
> +
> +	dev_dbg(dev, "streamed off Pass 1\n");
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> +
> +	if (enable) {
> +		/* Align vb2_core_streamon design */
> +		if (cam->streaming) {
> +			dev_warn(cam->dev, "already streaming on\n");
> +			return 0;
> +		}
> +		return mtk_cam_cio_stream_on(cam);
> +	}
> +
> +	if (!cam->streaming) {
> +		dev_warn(cam->dev, "already streaming off\n");
> +		return 0;
> +	}
> +	return mtk_cam_cio_stream_off(cam);
> +}
> +
> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> +				      struct v4l2_fh *fh,
> +				      struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static int mtk_cam_media_link_setup(struct media_entity *entity,
> +				    const struct media_pad *local,
> +				    const struct media_pad *remote, u32 flags)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> +	u32 pad = local->index;
> +
> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> +		__func__, pad, remote->index, flags);
> +
> +	/*
> +	 * The video nodes exposed by the driver have pads indexes
> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> +	 */
> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> +		cam->vdev_nodes[pad].enabled =
> +			!!(flags & MEDIA_LNK_FL_ENABLED);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	unsigned long flags;
> +
> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> +
> +	/* added the buffer into the tracking list */
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_add_tail(&buf->list, &node->buf_list);
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +
> +	/* update buffer internal address */
> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> +}
> +
> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct device *dev = cam->dev;
> +	struct mtk_cam_dev_buffer *buf;
> +	dma_addr_t addr;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	buf->node_id = node->id;
> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> +	buf->scp_addr = 0;
> +
> +	/* SCP address is only valid for meta input buffer */
> +	if (!node->desc.smem_alloc)
> +		return 0;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	/* Use coherent address to get iova address */
> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
> +	if (dma_mapping_error(dev, addr)) {
> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> +		return -EFAULT;
> +	}
> +	buf->scp_addr = buf->daddr;
> +	buf->daddr = addr;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> +		if (vb2_get_plane_payload(vb, 0) != size) {
> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> +				vb2_get_plane_payload(vb, 0), size);
> +			return -EINVAL;
> +		}
> +		return 0;
> +	}
> +
> +	v4l2_buf->field = V4L2_FIELD_NONE;
> +	vb2_set_plane_payload(vb, 0, size);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +	struct mtk_cam_dev_buffer *buf;
> +	struct device *dev = cam->dev;
> +
> +	if (!node->desc.smem_alloc)
> +		return;
> +
> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> +	dma_unmap_page_attrs(dev, buf->daddr,
> +			     vb->planes[0].length,
> +			     DMA_BIDIRECTIONAL,
> +			     DMA_ATTR_SKIP_CPU_SYNC);
> +}
> +
> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> +
> +	dev_dbg(cam->dev, "%s\n", __func__);
> +}
> +
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> +				   unsigned int *num_buffers,
> +				   unsigned int *num_planes,
> +				   unsigned int sizes[],
> +				   struct device *alloc_devs[])
> +{
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	unsigned int max_buffer_count = node->desc.max_buf_count;
> +	const struct v4l2_format *fmt = &node->vdev_fmt;
> +	unsigned int size;
> +
> +	/* Check the limitation of buffer size */
> +	if (max_buffer_count)
> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> +	if (node->desc.smem_alloc)
> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> +
> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		size = fmt->fmt.meta.buffersize;
> +	else
> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> +	if (*num_planes) {
> +		if (sizes[0] < size || *num_planes != 1)
> +			return -EINVAL;
> +	} else {
> +		*num_planes = 1;
> +		sizes[0] = size;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> +					   struct mtk_cam_video_device *node,
> +					   enum vb2_buffer_state state)
> +{
> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&node->buf_list_lock, flags);
> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> +	}
> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> +}
> +
> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> +				       unsigned int count)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!node->enabled) {
> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> +		ret = -ENOLINK;
> +		goto fail_ret_buf;
> +	}
> +
> +	mutex_lock(&cam->op_lock);
> +	/* Start streaming of the whole pipeline now*/
> +	if (!cam->pipeline.streaming_count) {
> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> +		if (ret) {
> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> +			goto fail_unlock;
> +		}
> +		mtk_cam_dev_init_stream(cam);
> +		ret = mtk_isp_hw_init(cam);
> +		if (ret) {
> +			dev_err(dev, "failed to init HW:%d\n", ret);
> +			goto fail_stop_pipeline;
> +		}
> +	}
> +
> +	/* Media links are fixed after media_pipeline_start */
> +	cam->stream_count++;
> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> +		cam->enabled_count);
> +	if (cam->stream_count < cam->enabled_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return 0;
> +	}
> +
> +	/* Stream on sub-devices node */
> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> +	if (ret)
> +		goto fail_no_stream;
> +	mutex_unlock(&cam->op_lock);
> +
> +	return 0;
> +
> +fail_no_stream:
> +	cam->stream_count--;
> +fail_stop_pipeline:
> +	if (cam->stream_count == 0)
> +		media_pipeline_stop(&node->vdev.entity);
> +fail_unlock:
> +	mutex_unlock(&cam->op_lock);
> +fail_ret_buf:
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> +	struct device *dev = cam->dev;
> +
> +	mutex_lock(&cam->op_lock);
> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> +		cam->stream_count);
> +	/* Check the first node to stream-off */
> +	if (cam->stream_count == cam->enabled_count)
> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> +
> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> +	cam->stream_count--;
> +	if (cam->stream_count) {
> +		mutex_unlock(&cam->op_lock);
> +		return;
> +	}
> +	mutex_unlock(&cam->op_lock);
> +
> +	mtk_cam_dev_req_cleanup(cam);
> +	media_pipeline_stop(&node->vdev.entity);
> +}
> +
> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +
> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> +		 dev_name(cam->dev));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index >= node->desc.num_fmts)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt = node->vdev_fmt.fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> +				  struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +	struct device *dev = cam->dev;
> +	const struct v4l2_format *dev_fmt;
> +	struct v4l2_format try_fmt;
> +
> +	memset(&try_fmt, 0, sizeof(try_fmt));
> +	try_fmt.type = f->type;
> +
> +	/* Validate pixelformat */
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> +	if (!dev_fmt) {
> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> +	}
> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> +
> +	/* Validate image width & height range */
> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> +	/* 4 bytes alignment for width */
> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> +
> +	/* Only support one plane */
> +	try_fmt.fmt.pix_mp.num_planes = 1;
> +
> +	/* bytesperline & sizeimage calculation */
> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> +
> +	/* Constant format fields */
> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +
> +	*f = try_fmt;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> +				struct v4l2_format *f)
> +{
> +	struct mtk_cam_dev *cam = video_drvdata(file);
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (vb2_is_busy(node->vdev.queue)) {
> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> +		return -EBUSY;
> +	}
> +
> +	/* Get the valid format */
> +	mtk_cam_vidioc_try_fmt(file, fh, f);
> +	/* Configure to video device */
> +	node->vdev_fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> +					  struct v4l2_frmsizeenum *sizes)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> +	const struct v4l2_format *dev_fmt;
> +
> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> +	if (!dev_fmt || sizes->index)
> +		return -EINVAL;
> +
> +	sizes->type = node->desc.frmsizes->type;
> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> +	       sizeof(sizes->stepwise));
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> +					struct v4l2_fmtdesc *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	if (f->index)
> +		return -EINVAL;
> +
> +	/* f->description is filled in v4l_fill_fmtdesc function */
> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->flags = 0;
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> +				     struct v4l2_format *f)
> +{
> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> +
> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> +	.subscribe_event = mtk_cam_sd_subscribe_event,
> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> +	.s_stream =  mtk_cam_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> +	.core = &mtk_cam_subdev_core_ops,
> +	.video = &mtk_cam_subdev_video_ops,
> +};
> +
> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> +	.link_setup = mtk_cam_media_link_setup,
> +	.link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct vb2_ops mtk_cam_vb2_ops = {
> +	.queue_setup = mtk_cam_vb2_queue_setup,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.buf_init = mtk_cam_vb2_buf_init,
> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> +	.start_streaming = mtk_cam_vb2_start_streaming,
> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> +	.buf_queue = mtk_cam_vb2_buf_queue,
> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> +	.buf_request_complete = mtk_cam_vb2_request_complete,
> +};
> +
> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static const struct media_device_ops mtk_cam_media_ops = {
> +	.req_alloc = mtk_cam_req_alloc,
> +	.req_free = mtk_cam_req_free,
> +	.req_validate = vb2_request_validate,
> +	.req_queue = mtk_cam_req_queue,
> +};
> +
> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> +				  struct media_device *media_dev)
> +{
> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	media_dev->dev = cam->dev;
> +	strscpy(media_dev->model, dev_driver_string(dev),
> +		sizeof(media_dev->model));
> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> +		 "platform:%s", dev_name(dev));
> +	media_dev->hw_revision = 0;
> +	media_device_init(media_dev);
> +	media_dev->ops = &mtk_cam_media_ops;
> +
> +	ret = media_device_register(media_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register media device:%d\n", ret);
> +		return ret;
> +	}
> +
> +	/* Initialize subdev pads */
> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> +					sizeof(*cam->subdev_pads),
> +					GFP_KERNEL);
> +	if (!cam->subdev_pads) {
> +		dev_err(dev, "failed to allocate subdev_pads\n");
> +		ret = -ENOMEM;
> +		goto fail_media_unreg;
> +	}
> +
> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> +				     cam->subdev_pads);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +
> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> +	for (i = 0; i < num_pads; i++)
> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	/* Customize the last one pad as CIO sink pad. */
> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> +	return 0;
> +
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int
> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> +			      struct mtk_cam_video_device *node)
> +{
> +	struct device *dev = cam->dev;
> +	struct video_device *vdev = &node->vdev;
> +	struct vb2_queue *vbq = &node->vbq;
> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> +	unsigned int link_flags = node->desc.link_flags;
> +	int ret;
> +
> +	/* Initialize mtk_cam_video_device */
> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> +		node->enabled = true;
> +	else
> +		node->enabled = false;
> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> +
> +	cam->subdev_pads[node->id].flags = output ?
> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> +
> +	/* Initialize media entities */
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> +		return ret;
> +	}
> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> +
> +	/* Initialize vbq */
> +	vbq->type = node->desc.buf_type;
> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> +		vbq->io_modes = VB2_MMAP;
> +	else
> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> +
> +	if (node->desc.smem_alloc) {
> +		vbq->bidirectional = 1;
> +		vbq->dev = cam->smem_dev;
> +	} else {
> +		vbq->dev = dev;
> +	}
> +	vbq->ops = &mtk_cam_vb2_ops;
> +	vbq->mem_ops = &vb2_dma_contig_memops;
> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> +	if (output)
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> +	else
> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> +	/* No minimum buffers limitation */
> +	vbq->min_buffers_needed = 0;
> +	vbq->drv_priv = cam;
> +	vbq->lock = &node->vdev_lock;
> +	vbq->supports_requests = true;
> +	vbq->requires_requests = true;
> +
> +	ret = vb2_queue_init(vbq);
> +	if (ret) {
> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> +		goto fail_media_clean;
> +	}
> +
> +	/* Initialize vdev */
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> +		 dev_driver_string(dev), node->desc.name);
> +	/* set cap/type/ioctl_ops of the video device */
> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> +	vdev->ioctl_ops = node->desc.ioctl_ops;
> +	vdev->fops = &mtk_cam_v4l2_fops;
> +	vdev->release = video_device_release_empty;
> +	vdev->lock = &node->vdev_lock;
> +	vdev->v4l2_dev = &cam->v4l2_dev;
> +	vdev->queue = &node->vbq;
> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> +	vdev->entity.ops = NULL;
> +	video_set_drvdata(vdev, cam);
> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> +
> +	/* Initialize miscellaneous variables */
> +	mutex_init(&node->vdev_lock);
> +	INIT_LIST_HEAD(&node->buf_list);
> +	spin_lock_init(&node->buf_list_lock);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret) {
> +		dev_err(dev, "failed to register vde:%d\n", ret);
> +		goto fail_vb2_rel;
> +	}
> +
> +	/* Create link between video node and the subdev pad */
> +	if (output) {
> +		ret = media_create_pad_link(&vdev->entity, 0,
> +					    &cam->subdev.entity,
> +					    node->id, link_flags);
> +	} else {
> +		ret = media_create_pad_link(&cam->subdev.entity,
> +					    node->id, &vdev->entity, 0,
> +					    link_flags);
> +	}
> +	if (ret)
> +		goto fail_vdev_ureg;
> +
> +	return 0;
> +
> +fail_vdev_ureg:
> +	video_unregister_device(vdev);
> +fail_vb2_rel:
> +	mutex_destroy(&node->vdev_lock);
> +	vb2_queue_release(vbq);
> +fail_media_clean:
> +	media_entity_cleanup(&vdev->entity);
> +
> +	return ret;
> +}
> +
> +static void
> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> +{
> +	video_unregister_device(&node->vdev);
> +	vb2_queue_release(&node->vbq);
> +	media_entity_cleanup(&node->vdev.entity);
> +	mutex_destroy(&node->vdev_lock);
> +}
> +
> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int i, ret;
> +
> +	/* Set up media device & pads */
> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> +	if (ret)
> +		return ret;
> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> +
> +	/* Set up v4l2 device */
> +	cam->v4l2_dev.mdev = &cam->media_dev;
> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> +		goto fail_media_unreg;
> +	}
> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> +
> +	/* Initialize subdev */
> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> +				V4L2_SUBDEV_FL_HAS_EVENTS;
> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> +		 "%s", dev_driver_string(dev));
> +	v4l2_set_subdevdata(&cam->subdev, cam);
> +
> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> +		goto fail_clean_media_entiy;
> +	}
> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> +
> +	/* Create video nodes and links */
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> +
> +		node->id = node->desc.id;
> +		ret = mtk_cam_video_register_device(cam, node);
> +		if (ret)
> +			goto fail_vdev_unreg;
> +	}
> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> +
> +	return 0;
> +
> +fail_vdev_unreg:
> +	for (i--; i >= 0; i--)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +fail_clean_media_entiy:
> +	media_entity_cleanup(&cam->subdev.entity);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +fail_media_unreg:
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return ret;
> +}
> +
> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> +{
> +	int i;
> +
> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> +
> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> +	v4l2_device_unregister_subdev(&cam->subdev);
> +	v4l2_device_unregister(&cam->v4l2_dev);
> +	media_entity_cleanup(&cam->subdev.entity);
> +	media_device_unregister(&cam->media_dev);
> +	media_device_cleanup(&cam->media_dev);
> +
> +	return 0;
> +}
> +
> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> +				      struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> +		return -ENODEV;
> +	}
> +
> +	cam->seninf = sd;
> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> +
> +	return 0;
> +}
> +
> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> +					struct v4l2_subdev *sd,
> +					struct v4l2_async_subdev *asd)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +
> +	cam->seninf = NULL;
> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> +}
> +
> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct mtk_cam_dev *cam =
> +		container_of(notifier, struct mtk_cam_dev, notifier);
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	if (!cam->seninf) {
> +		dev_err(dev, "No seninf subdev\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> +				    MEDIA_LNK_FL_IMMUTABLE |
> +				    MEDIA_LNK_FL_ENABLED);
> +	if (ret) {
> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> +			cam->seninf->entity.name, cam->subdev.entity.name,
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> +	if (ret) {
> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> +	.bound = mtk_cam_dev_notifier_bound,
> +	.unbind = mtk_cam_dev_notifier_unbind,
> +	.complete = mtk_cam_dev_notifier_complete,
> +};
> +
> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> +{
> +	struct device *dev = cam->dev;
> +	int ret;
> +
> +	v4l2_async_notifier_init(&cam->notifier);
> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> +	if (ret) {
> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> +		return ret;
> +	}
> +
> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> +	if (ret) {
> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> +		v4l2_async_notifier_cleanup(&cam->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> +{
> +	v4l2_async_notifier_unregister(&cam->notifier);
> +	v4l2_async_notifier_cleanup(&cam->notifier);
> +}
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf = vb2_ioctl_querybuf,
> +	.vidioc_qbuf = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> +	.vidioc_streamon = vb2_ioctl_streamon,
> +	.vidioc_streamoff = vb2_ioctl_streamoff,
> +	.vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static const struct v4l2_format meta_fmts[] = {
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> +			.buffersize = 512 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_3A,
> +			.buffersize = 1200 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_AF,
> +			.buffersize = 640 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> +			.buffersize = 288 * SZ_1K,
> +		},
> +	},
> +	{
> +		.fmt.meta = {
> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> +			.buffersize = 256,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format stream_out_fmts[] = {
> +	/* This is a default image format */
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> +		},
> +	},
> +};
> +
> +static const struct v4l2_format bin_out_fmts[] = {
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> +		},
> +	},
> +	{
> +		.fmt.pix_mp = {
> +			.width = IMG_MAX_WIDTH,
> +			.height = IMG_MAX_HEIGHT,
> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> +		},
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc output_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_META_IN_0,
> +		.name = "meta input",
> +		.cap = V4L2_CAP_META_OUTPUT,
> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = true,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 0,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> +	},
> +};
> +
> +static const struct
> +mtk_cam_dev_node_desc capture_queues[] = {
> +	{
> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> +		.name = "main stream",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_IMGO,
> +		.fmts = stream_out_fmts,
> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> +		.name = "packed out",
> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +		.link_flags = 0,
> +		.image = true,
> +		.smem_alloc = false,
> +		.dma_port = R_RRZO,
> +		.fmts = bin_out_fmts,
> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> +		.default_fmt_idx = 0,
> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> +		.frmsizes = &(struct v4l2_frmsizeenum) {
> +			.index = 0,
> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> +			.stepwise = {
> +				.max_width = IMG_MAX_WIDTH,
> +				.min_width = IMG_MIN_WIDTH,
> +				.max_height = IMG_MAX_HEIGHT,
> +				.min_height = IMG_MIN_HEIGHT,
> +				.step_height = 1,
> +				.step_width = 1,
> +			},
> +		},
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_0,
> +		.name = "partial meta 0",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AAO | R_FLKO | R_PSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 1,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_1,
> +		.name = "partial meta 1",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_AFO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 2,
> +		.max_buf_count = 5,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_2,
> +		.name = "partial meta 2",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LCSO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 3,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +	{
> +		.id = MTK_CAM_P1_META_OUT_3,
> +		.name = "partial meta 3",
> +		.cap = V4L2_CAP_META_CAPTURE,
> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> +		.link_flags = 0,
> +		.image = false,
> +		.smem_alloc = false,
> +		.dma_port = R_LMVO,
> +		.fmts = meta_fmts,
> +		.default_fmt_idx = 4,
> +		.max_buf_count = 10,
> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> +	},
> +};
> +
> +/* The helper to configure the device context */
> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> +{
> +	unsigned int node_idx;
> +	int i;
> +
> +	node_idx = 0;
> +	/* Setup the output queue */
> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> +
> +	/* Setup the capture queue */
> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> +}
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam)
> +{
> +	int ret;
> +
> +	cam->dev = &pdev->dev;
> +	mtk_cam_dev_queue_setup(cam);
> +
> +	spin_lock_init(&cam->pending_job_lock);
> +	spin_lock_init(&cam->running_job_lock);
> +	INIT_LIST_HEAD(&cam->pending_job_list);
> +	INIT_LIST_HEAD(&cam->running_job_list);
> +	mutex_init(&cam->op_lock);
> +
> +	/* v4l2 sub-device registration */
> +	ret = mtk_cam_v4l2_register(cam);
> +	if (ret)
> +		return ret;
> +
> +	ret = mtk_cam_v4l2_async_register(cam);
> +	if (ret)
> +		goto fail_v4l2_unreg;
> +
> +	return 0;
> +
> +fail_v4l2_unreg:
> +	mutex_destroy(&cam->op_lock);
> +	mtk_cam_v4l2_unregister(cam);
> +
> +	return ret;
> +}
> +
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> +{
> +	mtk_cam_v4l2_async_unregister(cam);
> +	mtk_cam_v4l2_unregister(cam);
> +	mutex_destroy(&cam->op_lock);
> +}
> +
> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> new file mode 100644
> index 000000000000..0a340a1e65ea
> --- /dev/null
> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> @@ -0,0 +1,244 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef __MTK_CAM_H__
> +#define __MTK_CAM_H__
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +#include <linux/platform_device.h>
> +#include <linux/spinlock.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "mtk_cam-ipi.h"
> +
> +#define IMG_MAX_WIDTH		5376
> +#define IMG_MAX_HEIGHT		4032
> +#define IMG_MIN_WIDTH		80
> +#define IMG_MIN_HEIGHT		60
> +
> +/*
> + * ID enum value for struct mtk_cam_dev_node_desc:id
> + * or mtk_cam_video_device:id
> + */
> +enum  {
> +	MTK_CAM_P1_META_IN_0 = 0,
> +	MTK_CAM_P1_MAIN_STREAM_OUT,
> +	MTK_CAM_P1_PACKED_BIN_OUT,
> +	MTK_CAM_P1_META_OUT_0,
> +	MTK_CAM_P1_META_OUT_1,
> +	MTK_CAM_P1_META_OUT_2,
> +	MTK_CAM_P1_META_OUT_3,
> +	MTK_CAM_P1_TOTAL_NODES
> +};
> +
> +/* Supported image format list */
> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> +
> +/* Supported bayer pixel order */
> +#define MTK_CAM_RAW_PXL_ID_B		0
> +#define MTK_CAM_RAW_PXL_ID_GB		1
> +#define MTK_CAM_RAW_PXL_ID_GR		2
> +#define MTK_CAM_RAW_PXL_ID_R		3
> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> +
> +/*
> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> + *
> + * @frame_seq_no: The frame sequence of frame in driver layer.
> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> + *
> + */
> +struct mtk_p1_frame_param {
> +	unsigned int frame_seq_no;
> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> +} __packed;
> +
> +/*
> + * struct mtk_cam_dev_request - MTK camera device request.
> + *
> + * @req: Embedded struct media request.
> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> + * @frame_work: work queue entry for frame transmission to SCP.
> + * @list: List entry of the object for @struct mtk_cam_dev:
> + *        pending_job_list or running_job_list.
> + * @timestamp: Start of frame timestamp in ns
> + *
> + */
> +struct mtk_cam_dev_request {
> +	struct media_request req;
> +	struct mtk_p1_frame_param frame_params;
> +	struct work_struct frame_work;
> +	struct list_head list;
> +	u64 timestamp;
> +};
> +
> +/*
> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> + *
> + * @vbb: Embedded struct vb2_v4l2_buffer.
> + * @list: List entry of the object for @struct mtk_cam_video_device:
> + *        buf_list.
> + * @daddr: The DMA address of this buffer.
> + * @scp_addr: The SCP address of this buffer which
> + *            is only supported for meta input node.
> + * @node_id: The vidoe node id which this buffer belongs to.
> + *
> + */
> +struct mtk_cam_dev_buffer {
> +	struct vb2_v4l2_buffer vbb;
> +	struct list_head list;
> +	/* Intenal part */
> +	dma_addr_t daddr;
> +	dma_addr_t scp_addr;
> +	unsigned int node_id;
> +};
> +
> +/*
> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> + *
> + * @id: id of the node
> + * @name: name of the node
> + * @cap: supported V4L2 capabilities
> + * @buf_type: supported V4L2 buffer type
> + * @dma_port: the dma ports associated to the node
> + * @link_flags: default media link flags
> + * @smem_alloc: using the smem_dev as alloc device or not
> + * @image: true for image node, false for meta node
> + * @num_fmts: the number of supported node formats
> + * @default_fmt_idx: default format of this node
> + * @max_buf_count: maximum VB2 buffer count
> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> + * @fmts: supported format
> + * @frmsizes: supported V4L2 frame size number
> + *
> + */
> +struct mtk_cam_dev_node_desc {
> +	u8 id;
> +	const char *name;
> +	u32 cap;
> +	u32 buf_type;
> +	u32 dma_port;
> +	u32 link_flags;
> +	u8 smem_alloc:1;
> +	u8 image:1;
> +	u8 num_fmts;
> +	u8 default_fmt_idx;
> +	u8 max_buf_count;
> +	const struct v4l2_ioctl_ops *ioctl_ops;
> +	const struct v4l2_format *fmts;
> +	const struct v4l2_frmsizeenum *frmsizes;
> +};
> +
> +/*
> + * struct mtk_cam_video_device - Mediatek video device structure
> + *
> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> + * @enabled: Indicate the video device is enabled or not
> + * @desc: The node description of video device
> + * @vdev_fmt: The V4L2 format of video device
> + * @vdev_pad: The media pad graph object of video device
> + * @vbq: A videobuf queue of video device
> + * @vdev: The video device instance
> + * @vdev_lock: Serializes vb2 queue and video device operations
> + * @buf_list: List for enqueue buffers
> + * @buf_list_lock: Lock used to protect buffer list.
> + *
> + */
> +struct mtk_cam_video_device {
> +	unsigned int id;
> +	unsigned int enabled;
> +	struct mtk_cam_dev_node_desc desc;
> +	struct v4l2_format vdev_fmt;
> +	struct media_pad vdev_pad;
> +	struct vb2_queue vbq;
> +	struct video_device vdev;
> +	/* Serializes vb2 queue and video device operations */
> +	struct mutex vdev_lock;
> +	struct list_head buf_list;
> +	/* Lock used to protect buffer list */
> +	spinlock_t buf_list_lock;
> +};
> +
> +/*
> + * struct mtk_cam_dev - Mediatek camera device structure.
> + *
> + * @dev: Pointer to device.
> + * @smem_pdev: Pointer to shared memory device.
> + * @pipeline: Media pipeline information.
> + * @media_dev: Media device instance.
> + * @subdev: The V4L2 sub-device instance.
> + * @v4l2_dev: The V4L2 device driver instance.
> + * @notifier: The v4l2_device notifier data.
> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> + * @seninf: Pointer to the seninf sub-device.
> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> + * @streaming: Indicate the overall streaming status is on or off.
> + * @enabled_dmas: The enabled dma port information when streaming on.
> + * @enabled_count: Number of enabled video nodes
> + * @stream_count: Number of streaming video nodes
> + * @running_job_count: Nunber of running jobs in the HW driver.
> + * @pending_job_list: List to keep the media requests before en-queue into
> + *                    HW driver.
> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> + * @running_job_list: List to keep the media requests after en-queue into
> + *                    HW driver.
> + * @running_job_lock: Protect the running_job_list data.
> + * @op_lock: Serializes driver's VB2 callback operations.
> + *
> + */
> +struct mtk_cam_dev {
> +	struct device *dev;
> +	struct device *smem_dev;
> +	struct media_pipeline pipeline;
> +	struct media_device media_dev;
> +	struct v4l2_subdev subdev;
> +	struct v4l2_device v4l2_dev;
> +	struct v4l2_async_notifier notifier;
> +	struct media_pad *subdev_pads;
> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> +	struct v4l2_subdev *seninf;
> +	struct v4l2_subdev *sensor;
> +	unsigned int streaming;
> +	unsigned int enabled_dmas;
> +	unsigned int enabled_count;
> +	unsigned int stream_count;
> +	unsigned int running_job_count;
> +	struct list_head pending_job_list;
> +	/* Protect the pending_job_list data */
> +	spinlock_t pending_job_lock;
> +	struct list_head running_job_list;
> +	/* Protect the running_job_list data & running_job_count */
> +	spinlock_t running_job_lock;
> +	/* Serializes driver's VB2 callback operations */
> +	struct mutex op_lock;
> +};
> +
> +int mtk_cam_dev_init(struct platform_device *pdev,
> +		     struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> +				   unsigned int frame_seq_no);
> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> +				  unsigned int frame_seq_no);
> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> +						unsigned int frame_seq_no);
> +
> +#endif /* __MTK_CAM_H__ */
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
  2019-12-19  5:49     ` Jungo Lin
  (?)
@ 2020-04-03  2:30       ` Laurent Pinchart
  -1 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2020-04-03  2:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

Hi Jungo,

Thank you for the patch.

On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> Add packed/full-g bayer formats with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.
> 
> (The current metadata format used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
>    v5.5-rc1 version.
> ---
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
>  8 files changed, 638 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> new file mode 100644
> index 000000000000..534edb4f0fd4
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> @@ -0,0 +1,65 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> +.. _v4l2-pix-fmt-mtisp-srggb10:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> +*******************************
> +
> +10-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +    * - start + 2:
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - G\ :sub:`03high bits 9--2`

This contradicts the description above, where you mention there's no
padding, and here only 8 bits are used for the two bytes. Which one is
correct ?

> +    * - start + 6:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +    * - start + 8:
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 10:
> +      - R\ :sub:`13high bits 9--2`
> +    * - start + 12:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +    * - start + 14:
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 16:
> +      - G\ :sub:`23high bits 9--2`
> +    * - start + 18:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +    * - start + 20:
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 22:
> +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> new file mode 100644
> index 000000000000..7be527711602
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> @@ -0,0 +1,90 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> +.. _v4l2-pix-fmt-mtisp-srggb10f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> +*******************************
> +
> +10-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.

I think this should describe where the additional green pixel comes
from.

> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - B\ :sub:`03high bits 9--2`
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`05high bits 3--0`
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 12:
> +      - G\ :sub:`13high bits 9--2`
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`15high bits 3--0`
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 20:
> +      - B\ :sub:`23high bits 9--2`
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`25high bits 3--0`
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 28:
> +      - G\ :sub:`33high bits 9--2`
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`35high bits 3--0`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> new file mode 100644
> index 000000000000..cc888aac42c2
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> @@ -0,0 +1,61 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> +.. _v4l2-pix-fmt-mtisp-srggb12:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> +*******************************
> +
> +12-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00lowbits 7--0`
> +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`01highbits 7--0`
> +      - B\ :sub:`02lowbits 7--0`
> +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`03highbits 7--0`
> +    * - start + 6:
> +      - G\ :sub:`10lowbits 7--0`
> +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`11highbits 7--0`
> +      - G\ :sub:`12lowbits 7--0`
> +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`13highbits 7--0`
> +    * - start + 12:
> +      - B\ :sub:`20lowbits 7--0`
> +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`21highbits 7--0`
> +      - B\ :sub:`22lowbits 7--0`
> +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`23highbits 7--0`
> +    * - start + 18:
> +      - G\ :sub:`30lowbits 7--0`
> +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`31highbits 7--0`
> +      - G\ :sub:`32lowbits 7--0`
> +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`33highbits 7--0`
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> new file mode 100644
> index 000000000000..c063de9f9ad8
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> +.. _v4l2-pix-fmt-mtisp-srggb12f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> +*******************************
> +
> +12-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> +    * - start + 2:
> +      - FG\ :sub:`01high bits 7--0`
> +      - G\ :sub:`02low bits 7--0`
> +    * - start + 4:
> +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`03high bits 7--0`
> +    * - start + 6:
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> +    * - start + 8:
> +      - G\ :sub:`05high bits 7--0`
> +      -
> +    * - start + 10:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> +    * - start + 12:
> +      - R\ :sub:`11high bits 7--0`
> +      - FG\ :sub:`12low bits 7--0`
> +    * - start + 14:
> +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`13high bits 7--0`
> +    * - start + 16:
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> +    * - start + 18:
> +      - FG\ :sub:`15high bits 7--0`
> +      -
> +    * - start + 20:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> +    * - start + 22:
> +      - FG\ :sub:`21high bits 7--0`
> +      - G\ :sub:`22low bits 7--0`
> +    * - start + 24:
> +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`23high bits 7--0`
> +    * - start + 26:
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`25high bits 7--0`
> +      -
> +    * - start + 30:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> +    * - start + 32:
> +      - R\ :sub:`31high bits 7--0`
> +      - FG\ :sub:`32low bits 7--0`
> +    * - start + 34:
> +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`33high bits 7--0`
> +    * - start + 36:
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> +    * - start + 38:
> +      - FG\ :sub:`35high bits 7--0`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> new file mode 100644
> index 000000000000..39ea9882a792
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> @@ -0,0 +1,73 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> +.. _v4l2-pix-fmt-mtisp-srggb14:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> +*******************************
> +
> +14-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`01low bits 9--2`\
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - B\ :sub:`02low bits 11--4`\
> +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`03high bits 13--6`\
> +      -
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`\
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 12:
> +      - G\ :sub:`12low bits 11--4`\
> +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`13high bits 13--6`\
> +      -
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`21low bits 9--2`\
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 20:
> +      - B\ :sub:`22low bits 11--4`\
> +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`23high bits 13--6`\
> +      -
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`\
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`32low bits 11--4`\
> +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`33high bits 13--6`\
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> new file mode 100644
> index 000000000000..010b1c190c60
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> +.. _v4l2-pix-fmt-mtisp-srggb14f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> +*******************************
> +
> +14-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`01low bits 9--2`
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - G\ :sub:`02low bits 11--4`
> +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`03high bits 13--6`
> +      - FG\ :sub:`04low bits 7--0`
> +    * - start + 8:
> +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`05high bits 9--2`
> +      - G\ :sub:`05high bits 13--10`
> +      -
> +    * - start + 12:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 16:
> +      - FG\ :sub:`12low bits 11--4`
> +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`13high bits 13--6`
> +      - R\ :sub:`14low bits 7--0`
> +    * - start + 20:
> +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`15high bits 9--2`
> +      - FG\ :sub:`15high bits 13--10`
> +      -
> +    * - start + 24:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`21low bits 9--2`
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`22low bits 11--4`
> +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`23high bits 13--6`
> +      - FG\ :sub:`24low bits 7--0`
> +    * - start + 32:
> +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`25high bits 9--2`
> +      - G\ :sub:`25high bits 13--10`
> +      -
> +    * - start + 36:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 40:
> +      - FG\ :sub:`32low bits 11--4`
> +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`33high bits 13--6`
> +      - R\ :sub:`34low bits 7--0`
> +    * - start + 44:
> +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`35high bits 9--2`
> +      - FG\ :sub:`35high bits 13--10`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> new file mode 100644
> index 000000000000..86cadbf38175
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> @@ -0,0 +1,51 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> +.. _v4l2-pix-fmt-mtisp-srggb8:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> +*******************************
> +
> +8-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.

How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
other variants) ? They seem identical based on the description.

> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - G\ :sub:`01`
> +      - B\ :sub:`02`
> +      - G\ :sub:`03`
> +    * - start + 4:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - G\ :sub:`12`
> +      - R\ :sub:`13`
> +    * - start + 8:
> +      - B\ :sub:`20`
> +      - G\ :sub:`21`
> +      - B\ :sub:`22`
> +      - G\ :sub:`23`
> +    * - start + 12:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - G\ :sub:`32`
> +      - R\ :sub:`33`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> new file mode 100644
> index 000000000000..ca5151312bca
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> @@ -0,0 +1,78 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> +.. _v4l2-pix-fmt-mtisp-srggb8f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> +*******************************
> +
> +8-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - start + 6:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +    * - start + 12:
> +      - B\ :sub:`20`
> +      - FG\ :sub:`21`
> +      - G\ :sub:`22`
> +      - B\ :sub:`23`
> +      - FG\ :sub:`24`
> +      - G\ :sub:`25`
> +    * - start + 18:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - FG\ :sub:`32`
> +      - G\ :sub:`33`
> +      - R\ :sub:`34`
> +      - FG\ :sub:`35`
> \ No newline at end of file

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2020-04-03  2:30       ` Laurent Pinchart
  0 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2020-04-03  2:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, robh, Rynn.Wu, suleiman, Jerry-ch.Chen,
	frederic.chen, linux-media, devicetree, hverkuil-cisco, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi Jungo,

Thank you for the patch.

On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> Add packed/full-g bayer formats with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.
> 
> (The current metadata format used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
>    v5.5-rc1 version.
> ---
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
>  8 files changed, 638 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> new file mode 100644
> index 000000000000..534edb4f0fd4
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> @@ -0,0 +1,65 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> +.. _v4l2-pix-fmt-mtisp-srggb10:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> +*******************************
> +
> +10-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +    * - start + 2:
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - G\ :sub:`03high bits 9--2`

This contradicts the description above, where you mention there's no
padding, and here only 8 bits are used for the two bytes. Which one is
correct ?

> +    * - start + 6:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +    * - start + 8:
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 10:
> +      - R\ :sub:`13high bits 9--2`
> +    * - start + 12:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +    * - start + 14:
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 16:
> +      - G\ :sub:`23high bits 9--2`
> +    * - start + 18:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +    * - start + 20:
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 22:
> +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> new file mode 100644
> index 000000000000..7be527711602
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> @@ -0,0 +1,90 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> +.. _v4l2-pix-fmt-mtisp-srggb10f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> +*******************************
> +
> +10-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.

I think this should describe where the additional green pixel comes
from.

> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - B\ :sub:`03high bits 9--2`
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`05high bits 3--0`
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 12:
> +      - G\ :sub:`13high bits 9--2`
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`15high bits 3--0`
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 20:
> +      - B\ :sub:`23high bits 9--2`
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`25high bits 3--0`
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 28:
> +      - G\ :sub:`33high bits 9--2`
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`35high bits 3--0`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> new file mode 100644
> index 000000000000..cc888aac42c2
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> @@ -0,0 +1,61 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> +.. _v4l2-pix-fmt-mtisp-srggb12:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> +*******************************
> +
> +12-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00lowbits 7--0`
> +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`01highbits 7--0`
> +      - B\ :sub:`02lowbits 7--0`
> +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`03highbits 7--0`
> +    * - start + 6:
> +      - G\ :sub:`10lowbits 7--0`
> +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`11highbits 7--0`
> +      - G\ :sub:`12lowbits 7--0`
> +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`13highbits 7--0`
> +    * - start + 12:
> +      - B\ :sub:`20lowbits 7--0`
> +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`21highbits 7--0`
> +      - B\ :sub:`22lowbits 7--0`
> +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`23highbits 7--0`
> +    * - start + 18:
> +      - G\ :sub:`30lowbits 7--0`
> +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`31highbits 7--0`
> +      - G\ :sub:`32lowbits 7--0`
> +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`33highbits 7--0`
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> new file mode 100644
> index 000000000000..c063de9f9ad8
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> +.. _v4l2-pix-fmt-mtisp-srggb12f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> +*******************************
> +
> +12-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> +    * - start + 2:
> +      - FG\ :sub:`01high bits 7--0`
> +      - G\ :sub:`02low bits 7--0`
> +    * - start + 4:
> +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`03high bits 7--0`
> +    * - start + 6:
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> +    * - start + 8:
> +      - G\ :sub:`05high bits 7--0`
> +      -
> +    * - start + 10:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> +    * - start + 12:
> +      - R\ :sub:`11high bits 7--0`
> +      - FG\ :sub:`12low bits 7--0`
> +    * - start + 14:
> +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`13high bits 7--0`
> +    * - start + 16:
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> +    * - start + 18:
> +      - FG\ :sub:`15high bits 7--0`
> +      -
> +    * - start + 20:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> +    * - start + 22:
> +      - FG\ :sub:`21high bits 7--0`
> +      - G\ :sub:`22low bits 7--0`
> +    * - start + 24:
> +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`23high bits 7--0`
> +    * - start + 26:
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`25high bits 7--0`
> +      -
> +    * - start + 30:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> +    * - start + 32:
> +      - R\ :sub:`31high bits 7--0`
> +      - FG\ :sub:`32low bits 7--0`
> +    * - start + 34:
> +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`33high bits 7--0`
> +    * - start + 36:
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> +    * - start + 38:
> +      - FG\ :sub:`35high bits 7--0`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> new file mode 100644
> index 000000000000..39ea9882a792
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> @@ -0,0 +1,73 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> +.. _v4l2-pix-fmt-mtisp-srggb14:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> +*******************************
> +
> +14-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`01low bits 9--2`\
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - B\ :sub:`02low bits 11--4`\
> +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`03high bits 13--6`\
> +      -
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`\
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 12:
> +      - G\ :sub:`12low bits 11--4`\
> +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`13high bits 13--6`\
> +      -
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`21low bits 9--2`\
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 20:
> +      - B\ :sub:`22low bits 11--4`\
> +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`23high bits 13--6`\
> +      -
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`\
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`32low bits 11--4`\
> +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`33high bits 13--6`\
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> new file mode 100644
> index 000000000000..010b1c190c60
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> +.. _v4l2-pix-fmt-mtisp-srggb14f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> +*******************************
> +
> +14-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`01low bits 9--2`
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - G\ :sub:`02low bits 11--4`
> +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`03high bits 13--6`
> +      - FG\ :sub:`04low bits 7--0`
> +    * - start + 8:
> +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`05high bits 9--2`
> +      - G\ :sub:`05high bits 13--10`
> +      -
> +    * - start + 12:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 16:
> +      - FG\ :sub:`12low bits 11--4`
> +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`13high bits 13--6`
> +      - R\ :sub:`14low bits 7--0`
> +    * - start + 20:
> +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`15high bits 9--2`
> +      - FG\ :sub:`15high bits 13--10`
> +      -
> +    * - start + 24:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`21low bits 9--2`
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`22low bits 11--4`
> +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`23high bits 13--6`
> +      - FG\ :sub:`24low bits 7--0`
> +    * - start + 32:
> +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`25high bits 9--2`
> +      - G\ :sub:`25high bits 13--10`
> +      -
> +    * - start + 36:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 40:
> +      - FG\ :sub:`32low bits 11--4`
> +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`33high bits 13--6`
> +      - R\ :sub:`34low bits 7--0`
> +    * - start + 44:
> +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`35high bits 9--2`
> +      - FG\ :sub:`35high bits 13--10`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> new file mode 100644
> index 000000000000..86cadbf38175
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> @@ -0,0 +1,51 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> +.. _v4l2-pix-fmt-mtisp-srggb8:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> +*******************************
> +
> +8-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.

How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
other variants) ? They seem identical based on the description.

> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - G\ :sub:`01`
> +      - B\ :sub:`02`
> +      - G\ :sub:`03`
> +    * - start + 4:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - G\ :sub:`12`
> +      - R\ :sub:`13`
> +    * - start + 8:
> +      - B\ :sub:`20`
> +      - G\ :sub:`21`
> +      - B\ :sub:`22`
> +      - G\ :sub:`23`
> +    * - start + 12:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - G\ :sub:`32`
> +      - R\ :sub:`33`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> new file mode 100644
> index 000000000000..ca5151312bca
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> @@ -0,0 +1,78 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> +.. _v4l2-pix-fmt-mtisp-srggb8f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> +*******************************
> +
> +8-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - start + 6:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +    * - start + 12:
> +      - B\ :sub:`20`
> +      - FG\ :sub:`21`
> +      - G\ :sub:`22`
> +      - B\ :sub:`23`
> +      - FG\ :sub:`24`
> +      - G\ :sub:`25`
> +    * - start + 18:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - FG\ :sub:`32`
> +      - G\ :sub:`33`
> +      - R\ :sub:`34`
> +      - FG\ :sub:`35`
> \ No newline at end of file

-- 
Regards,

Laurent Pinchart

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2020-04-03  2:30       ` Laurent Pinchart
  0 siblings, 0 replies; 388+ messages in thread
From: Laurent Pinchart @ 2020-04-03  2:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, robh, Rynn.Wu, suleiman, Jerry-ch.Chen,
	frederic.chen, linux-media, devicetree, hverkuil-cisco, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi Jungo,

Thank you for the patch.

On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> Add packed/full-g bayer formats with 8/10/12/14 bit
> for image output. Add Pass 1 (P1) specific meta formats for
> parameter processing and 3A/other statistics.
> 
> (The current metadata format used in meta input and partial
> meta nodes is only a temporary solution to kick off the driver
> development and is not ready to be reviewed yet.)
> 
> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> ---
> Changes from v6:
>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
>    v5.5-rc1 version.
> ---
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
>  8 files changed, 638 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> new file mode 100644
> index 000000000000..534edb4f0fd4
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> @@ -0,0 +1,65 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> +.. _v4l2-pix-fmt-mtisp-srggb10:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> +*******************************
> +
> +10-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +    * - start + 2:
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - G\ :sub:`03high bits 9--2`

This contradicts the description above, where you mention there's no
padding, and here only 8 bits are used for the two bytes. Which one is
correct ?

> +    * - start + 6:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +    * - start + 8:
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 10:
> +      - R\ :sub:`13high bits 9--2`
> +    * - start + 12:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +    * - start + 14:
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 16:
> +      - G\ :sub:`23high bits 9--2`
> +    * - start + 18:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +    * - start + 20:
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 22:
> +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> new file mode 100644
> index 000000000000..7be527711602
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> @@ -0,0 +1,90 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> +.. _v4l2-pix-fmt-mtisp-srggb10f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> +*******************************
> +
> +10-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.

I think this should describe where the additional green pixel comes
from.

> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> +    * - start + 4:
> +      - B\ :sub:`03high bits 9--2`
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`05high bits 3--0`
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> +    * - start + 12:
> +      - G\ :sub:`13high bits 9--2`
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`15high bits 3--0`
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> +    * - start + 20:
> +      - B\ :sub:`23high bits 9--2`
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> +      - G\ :sub:`25high bits 3--0`
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> +    * - start + 28:
> +      - G\ :sub:`33high bits 9--2`
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> +      - FG\ :sub:`35high bits 3--0`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> new file mode 100644
> index 000000000000..cc888aac42c2
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> @@ -0,0 +1,61 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> +.. _v4l2-pix-fmt-mtisp-srggb12:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> +*******************************
> +
> +12-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00lowbits 7--0`
> +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`01highbits 7--0`
> +      - B\ :sub:`02lowbits 7--0`
> +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`03highbits 7--0`
> +    * - start + 6:
> +      - G\ :sub:`10lowbits 7--0`
> +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`11highbits 7--0`
> +      - G\ :sub:`12lowbits 7--0`
> +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`13highbits 7--0`
> +    * - start + 12:
> +      - B\ :sub:`20lowbits 7--0`
> +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`21highbits 7--0`
> +      - B\ :sub:`22lowbits 7--0`
> +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> +      - G\ :sub:`23highbits 7--0`
> +    * - start + 18:
> +      - G\ :sub:`30lowbits 7--0`
> +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`31highbits 7--0`
> +      - G\ :sub:`32lowbits 7--0`
> +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> +      - R\ :sub:`33highbits 7--0`
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> new file mode 100644
> index 000000000000..c063de9f9ad8
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> +.. _v4l2-pix-fmt-mtisp-srggb12f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> +*******************************
> +
> +12-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> +    * - start + 2:
> +      - FG\ :sub:`01high bits 7--0`
> +      - G\ :sub:`02low bits 7--0`
> +    * - start + 4:
> +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`03high bits 7--0`
> +    * - start + 6:
> +      - FG\ :sub:`04low bits 7--0`
> +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> +    * - start + 8:
> +      - G\ :sub:`05high bits 7--0`
> +      -
> +    * - start + 10:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> +    * - start + 12:
> +      - R\ :sub:`11high bits 7--0`
> +      - FG\ :sub:`12low bits 7--0`
> +    * - start + 14:
> +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`13high bits 7--0`
> +    * - start + 16:
> +      - R\ :sub:`14low bits 7--0`
> +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> +    * - start + 18:
> +      - FG\ :sub:`15high bits 7--0`
> +      -
> +    * - start + 20:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> +    * - start + 22:
> +      - FG\ :sub:`21high bits 7--0`
> +      - G\ :sub:`22low bits 7--0`
> +    * - start + 24:
> +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> +      - B\ :sub:`23high bits 7--0`
> +    * - start + 26:
> +      - FG\ :sub:`24low bits 7--0`
> +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`25high bits 7--0`
> +      -
> +    * - start + 30:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> +    * - start + 32:
> +      - R\ :sub:`31high bits 7--0`
> +      - FG\ :sub:`32low bits 7--0`
> +    * - start + 34:
> +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> +      - G\ :sub:`33high bits 7--0`
> +    * - start + 36:
> +      - R\ :sub:`34low bits 7--0`
> +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> +    * - start + 38:
> +      - FG\ :sub:`35high bits 7--0`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> new file mode 100644
> index 000000000000..39ea9882a792
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> @@ -0,0 +1,73 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> +.. _v4l2-pix-fmt-mtisp-srggb14:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> +*******************************
> +
> +14-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`01low bits 9--2`\
> +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - B\ :sub:`02low bits 11--4`\
> +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`03high bits 13--6`\
> +      -
> +    * - start + 8:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`\
> +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 12:
> +      - G\ :sub:`12low bits 11--4`\
> +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`13high bits 13--6`\
> +      -
> +    * - start + 16:
> +      - B\ :sub:`20low bits 7--0`
> +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`21low bits 9--2`\
> +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 20:
> +      - B\ :sub:`22low bits 11--4`\
> +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`23high bits 13--6`\
> +      -
> +    * - start + 24:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`\
> +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`32low bits 11--4`\
> +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - R\ :sub:`33high bits 13--6`\
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> new file mode 100644
> index 000000000000..010b1c190c60
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> @@ -0,0 +1,110 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> +.. _v4l2-pix-fmt-mtisp-srggb14f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> +*******************************
> +
> +14-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00low bits 7--0`
> +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`01low bits 9--2`
> +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> +    * - start + 4:
> +      - G\ :sub:`02low bits 11--4`
> +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`03high bits 13--6`
> +      - FG\ :sub:`04low bits 7--0`
> +    * - start + 8:
> +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`05high bits 9--2`
> +      - G\ :sub:`05high bits 13--10`
> +      -
> +    * - start + 12:
> +      - G\ :sub:`10low bits 7--0`
> +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`11low bits 9--2`
> +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> +    * - start + 16:
> +      - FG\ :sub:`12low bits 11--4`
> +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`13high bits 13--6`
> +      - R\ :sub:`14low bits 7--0`
> +    * - start + 20:
> +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`15high bits 9--2`
> +      - FG\ :sub:`15high bits 13--10`
> +      -
> +    * - start + 24:
> +      - B\ :sub:`20low bits 7--0`
> +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`21low bits 9--2`
> +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> +    * - start + 28:
> +      - G\ :sub:`22low bits 11--4`
> +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> +      - B\ :sub:`23high bits 13--6`
> +      - FG\ :sub:`24low bits 7--0`
> +    * - start + 32:
> +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> +      - G\ :sub:`25high bits 9--2`
> +      - G\ :sub:`25high bits 13--10`
> +      -
> +    * - start + 36:
> +      - G\ :sub:`30low bits 7--0`
> +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> +      - R\ :sub:`31low bits 9--2`
> +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> +    * - start + 40:
> +      - FG\ :sub:`32low bits 11--4`
> +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> +      - G\ :sub:`33high bits 13--6`
> +      - R\ :sub:`34low bits 7--0`
> +    * - start + 44:
> +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> +      - FG\ :sub:`35high bits 9--2`
> +      - FG\ :sub:`35high bits 13--10`
> +      -
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> new file mode 100644
> index 000000000000..86cadbf38175
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> @@ -0,0 +1,51 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> +.. _v4l2-pix-fmt-mtisp-srggb8:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> +*******************************
> +
> +8-bit Packed Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format, meaning all the data bits for a pixel lying
> +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor.
> +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> +Below is an example of conventional RGB byte order BGGR.

How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
other variants) ? They seem identical based on the description.

> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - G\ :sub:`01`
> +      - B\ :sub:`02`
> +      - G\ :sub:`03`
> +    * - start + 4:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - G\ :sub:`12`
> +      - R\ :sub:`13`
> +    * - start + 8:
> +      - B\ :sub:`20`
> +      - G\ :sub:`21`
> +      - B\ :sub:`22`
> +      - G\ :sub:`23`
> +    * - start + 12:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - G\ :sub:`32`
> +      - R\ :sub:`33`
> \ No newline at end of file
> diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> new file mode 100644
> index 000000000000..ca5151312bca
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> @@ -0,0 +1,78 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> +.. _v4l2-pix-fmt-mtisp-srggb8f:
> +
> +*******************************
> +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> +*******************************
> +
> +8-bit Packed Full-G Bayer formats.
> +
> +Description
> +===========
> +
> +These four pixel formats are used by Mediatek ISP P1.
> +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> +Full-G means 1 more pixel for green channel every 2 pixels.
> +The least significant byte is stored at lower memory addresses (little-endian).
> +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> +RGB byte order BGGR.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +
> +**Byte Order.**
> +Each cell is one byte.
> +
> +.. flat-table::
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - B\ :sub:`00`
> +      - FG\ :sub:`01`
> +      - G\ :sub:`02`
> +      - B\ :sub:`03`
> +      - FG\ :sub:`04`
> +      - G\ :sub:`05`
> +    * - start + 6:
> +      - G\ :sub:`10`
> +      - R\ :sub:`11`
> +      - FG\ :sub:`12`
> +      - G\ :sub:`13`
> +      - R\ :sub:`14`
> +      - FG\ :sub:`15`
> +    * - start + 12:
> +      - B\ :sub:`20`
> +      - FG\ :sub:`21`
> +      - G\ :sub:`22`
> +      - B\ :sub:`23`
> +      - FG\ :sub:`24`
> +      - G\ :sub:`25`
> +    * - start + 18:
> +      - G\ :sub:`30`
> +      - R\ :sub:`31`
> +      - FG\ :sub:`32`
> +      - G\ :sub:`33`
> +      - R\ :sub:`34`
> +      - FG\ :sub:`35`
> \ No newline at end of file

-- 
Regards,

Laurent Pinchart

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2020-03-31 15:34       ` Helen Koike
@ 2020-04-09  2:05         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-09  2:05 UTC (permalink / raw)
  To: Helen Koike
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hi Helen:

Thanks for your comments.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hello Jungo,
> 
> I was taking a look at this patch (thanks for the work),
> I didn't look in deep details, but I have some comments, please see
> below. I hope it helps.
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >  - Correct auto suspend timer value for suspend/resume issue
> >  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >  - Fix KE due to no sen-inf sub-device
> > ---
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> 
> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
> capture, which for metadata, and what is being used to deal with requests.
> 
> It would make it easier to review imho.
> 

For file structure design, it was reviewed in the previous patch
serials.
e.g.
https://patchwork.kernel.org/patch/10938137/
If you think it is better, I will modify it.

> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> 
> It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.
> 
> Or maybe something like:
> 
> mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
> mtkisp_p1_capture.c
> mtkisp_p1_meta.c
> mtkisp_p1_isp.c
> mtkisp_p1_hw.c (or maybe split this between the other files)
> mtkisp_p1_request.c
> mtkisp_p1_common.c (?)
> 
> or s/mtkisp_p1/mtk_cam/
> 
> what do you think?
> 

Ok, I will revise our naming issue for consistency reason.

> >  9 files changed, 3377 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> > new file mode 100644
> > index 000000000000..f86e1b59ad1e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/Kconfig
> > @@ -0,0 +1,20 @@
> > +config VIDEO_MEDIATEK_ISP_PASS1
> > +	tristate "Mediatek ISP Pass 1 driver"
> > +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> 
> I think you need OF as well
> 
> depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF
> 
> > +	depends on ARCH_MEDIATEK
> 
> depends on ARCH_MEDIATEK || COMPILE_TEST
> 

Ok, we will fix this in next patch.

> > +	select V4L2_FWNODE
> > +	select VIDEOBUF2_VMALLOC
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select MTK_SCP
> > +	default n
> > +	help
> > +		Pass 1 driver controls 3A (auto-focus, exposure,
> > +		and white balance) with tuning feature and outputs
> > +		the captured image buffers in Mediatek's camera system.
> > +
> > +		Choose Y if you want to use Mediatek SoCs to create image
> > +		captured application such as video recording and still image
> > +		capturing.
> 
> I would re-word this a bit, since people can use a captured application (and not create one) :)
> 

Ok, I will re-word as "if you want to use image captured application
based on Mediatek SoCs for video recording and still image capturing
functions"

> > +
> > +		To compile this driver as a module, choose M here; the module
> > +		will be called mtk-cam-isp.
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > new file mode 100644
> > index 000000000000..ce79d283b209
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..53b54d3c26a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +mtk-cam-isp-objs += mtk_cam.o
> > +mtk-cam-isp-objs += mtk_cam-hw.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > new file mode 100644
> > index 000000000000..4065d0d29b7f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > @@ -0,0 +1,636 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/module.h>
> > +#include <linux/remoteproc/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include <media/v4l2-event.h>
> 
> Please sort headers alphabetically.
> 

Will fix in next patch.

> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +#include "mtk_cam-regs.h"
> > +
> > +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> > +#define MTK_ISP_CQ_BUFFER_COUNT			3
> > +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> > +
> > +/*
> > + *
> > + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> > + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> > + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> > + * In current driver, only supports CAM B.
> > + *
> > + */
> > +#define MTK_ISP_CAM_ID_B			3
> > +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> > +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> > +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> > +
> > +static void isp_tx_frame_worker(struct work_struct *work)
> 
> I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
> an external function.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > +
> > +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> > +		dev_err(dev, "wrong IPI len:%d\n", len);
> > +		return;
> > +	}
> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> > +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> > +		return;
> > +
> > +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> > +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> > +}
> > +
> > +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI cmd\n");
> > +		return ret;
> > +	}
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI frame\n");
> > +		goto unreg_ipi_cmd;
> > +	}
> > +
> > +	p1_dev->composer_wq =
> > +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> > +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> > +					WQ_FREEZABLE);
> > +	if (!p1_dev->composer_wq) {
> > +		dev_err(dev, "failed to alloc composer workqueue\n");
> > +		goto unreg_ipi_frame;
> > +	}
> > +
> > +	return 0;
> > +
> > +unreg_ipi_frame:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +unreg_ipi_cmd:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	destroy_workqueue(p1_dev->composer_wq);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +}
> > +
> > +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> > +
> > +	/*
> > +	 * Passed coherent reserved memory info. for SCP firmware usage.
> > +	 * This buffer is used for SCP's ISP composer to compose.
> > +	 * The size of is fixed to 0x200000 for the requirement of composer.
> > +	 */
> > +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> > +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +
> > +	isp_composer_uninit(p1_dev);
> 
> I think you can copy the 3 lines of this isp_composer_uninit() function here, since
> this seems the only place it is being used, and having a deinit and uninit function is
> a bit confusing.
> 

Fix in next patch.

> > +}
> > +
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> > +		       struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> 
> I prefer not having a int parameter, this is easier to read:
> 
> mtk_isp_stream_on(cam);
> mtk_isp_stream_off(cam);
> 
> or
> 
> mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
> mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);
> 
> instead of:
> 
> mtk_isp_stream(cam, 1);
> mtk_isp_stream(cam, 0);
> 
> You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.
> 

Ok, I will choose the method 2.

> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> 
> s/is_stream_on/is_streaming
> 

Fix in next patch.

> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		dev_err(dev, "failed to rproc_boot\n");
> 
> It would be nice to improve this error message for users, how about:
> 
> dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);
> 
> Or maybe even remove this message, since rproc_boot() already have several error messages.
> 

Ok, we will remove the error message.

> > +		return ret;
> > +	}
> > +
> > +	ret = isp_composer_init(p1_dev);
> > +	if (ret)
> 
> should rproc_shutdown() be called here?
> 

Yes, we will fix it.

> > +		return ret;
> > +
> > +	pm_runtime_get_sync(dev);
> 
> You should check return value here.
> 

Fix in next patch.

> > +	isp_composer_hw_init(p1_dev);
> > +
> > +	p1_dev->enqueued_frame_seq_no = 0;
> > +	p1_dev->dequeued_frame_seq_no = 0;
> > +	p1_dev->composed_frame_seq_no = 0;
> > +	p1_dev->sof_count = 0;
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	isp_composer_hw_deinit(p1_dev);
> > +	pm_runtime_mark_last_busy(dev);
> > +	pm_runtime_put_autosuspend(dev);
> > +	rproc_shutdown(p1_dev->rproc_handle);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> > +
> > +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> > +			       unsigned int dequeued_frame_seq_no)
> > +{
> > +	dma_addr_t base_addr = p1_dev->composer_iova;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_cam_dev_request *req;
> > +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> > +	unsigned int addr_offset;
> > +
> > +	/* Send V4L2_EVENT_FRAME_SYNC event */
> > +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +
> > +	p1_dev->sof_count += 1;
> > +	/* Save frame information */
> > +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> > +
> > +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +	if (req)
> > +		req->timestamp = ktime_get_boottime_ns();
> > +
> > +	/* Update CQ base address if needed */
> > +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> > +		dev_dbg(dev,
> > +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> > +			composed_frame_seq_no, dequeued_frame_seq_no);
> > +		return;
> > +	}
> > +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> > +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> > +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> > +}
> > +
> > +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	u32 val;
> > +
> > +	dev_err(p1_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(p1_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> 
> I think if would be better to transfor those into dev_dbg and add a counter
> in debugfs.
> 

These error messages are important for debugging.
I suggest to keep in dev_err.

Moreover, could you give more information about debug counter?
I don't get your point.
Do you suggest to accumulate the total count of DMA errors?

> > +
> > +	/* Disable DMA error mask to avoid too much error log */
> > +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> > +}
> > +
> > +static irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> > +	struct device *dev = p1_dev->dev;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> > +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> > +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> > +
> > +	/*
> > +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> > +	 * If these two ISRs come together, print warning msg to hint.
> > +	 */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> > +
> > +	/* De-queue frame */
> > +	if (irq_status & SW_PASS1_DON_ST) {
> 
> I suppose this means "done streaming"?
> 

Yes, it means the frame buffer is outputed done.

> > +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> > +					      p1_dev->dequeued_frame_seq_no);
> > +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> > +	}
> > +
> > +	/* Save frame info. & update CQ address for frame HW en-queue */
> > +	if (irq_status & SOF_INT_ST)
> > +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_irq_handle_dma_err(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> > +		p1_dev->sof_count, irq_status, dma_status,
> > +		dequeued_frame_seq_no);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> > +			       struct platform_device *pdev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	dma_addr_t addr;
> > +	void *ptr;
> 
> Maybe "composer_buffer" would be a better name.
> 
> But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?
> 

Ok, I will use p1_dev->composer_virt_addr directly.

> > +	int ret;
> > +
> > +	p1_dev->scp = scp_get(pdev);
> > +	if (!p1_dev->scp) {
> > +		dev_err(dev, "failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> > +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> 
> I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
> which mapping we are refering to.
> 

Fix in next patch.

> > +
> > +	/*
> > +	 * Allocate coherent reserved memory for SCP firmware usage.
> > +	 * The size of SCP composer's memory is fixed to 0x200000
> > +	 * for the requirement of firmware.
> > +	 */
> > +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> > +	if (!ptr) {
> > +		ret = -ENOMEM;
> > +		goto fail_put_scp;
> > +	}
> > +
> > +	p1_dev->composer_scp_addr = addr;
> > +	p1_dev->composer_virt_addr = ptr;
> > +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> > +
> > +	/*
> > +	 * This reserved memory is also be used by ISP P1 HW.
> > +	 * Need to get iova address for ISP P1 DMA.
> > +	 */
> > +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> > +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map scp iova\n");
> > +		ret = -ENOMEM;
> > +		goto fail_free_mem;
> > +	}
> > +	p1_dev->composer_iova = addr;
> 
> why not rename this to composer_isp_addr ?
> Since, afaik, composer_scp_addr is also iova.
> 
> At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).
> 

Ok, we will rename composer_iova to composer_isp_addr.
Basically, scp_addr is reserved physical address and it is not behind an
IOMMU.

> > +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> > +
> > +	return 0;
> > +
> > +fail_free_mem:
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +fail_put_scp:
> > +	scp_put(p1_dev->scp);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +	scp_put(p1_dev->scp);
> > +}
> > +
> > +static int mtk_isp_pm_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Disable ISP's view finder and wait for TG idle if possible */
> > +	dev_dbg(dev, "cam suspend, disable VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> > +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> > +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> > +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> > +
> > +	/* Disable CMOS */
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Force ISP HW to idle */
> > +	ret = pm_runtime_force_suspend(dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to force suspend:%d\n", ret);
> > +		goto reenable_hw;
> > +	}
> > +
> > +	return 0;
> > +
> > +reenable_hw:
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_pm_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Force ISP HW to resume */
> > +	ret = pm_runtime_force_resume(dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Enable CMOS */
> > +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Enable VF */
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	dev_dbg(dev, "%s:disable clock\n", __func__);
> > +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s:enable clock\n", __func__);
> > +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to enable clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	/* List of clocks required by isp cam */
> > +	static const char * const clk_names[] = {
> > +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +	};
> > +	struct mtk_isp_p1_device *p1_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq, ret, i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->dev = dev;
> > +	dev_set_drvdata(dev, p1_dev);
> > +
> > +	/*
> > +	 * Now only support single CAM with CAM B.
> > +	 * Get CAM B register base with CAM B index.
> > +	 * Support multiple CAMs in future.
> > +	 */
> > +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> > +	p1_dev->regs = devm_ioremap_resource(dev, res);
> > +	if (IS_ERR(p1_dev->regs)) {
> > +		dev_err(dev, "failed to map reister base\n");
> 
> s/reister/register
> 

Fix in next patch.

> > +		return PTR_ERR(p1_dev->regs);
> > +	}
> > +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> > +
> > +	/*
> > +	 * The cam_sys unit only supports reg., but has no IRQ support.
> > +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> > +	 */
> > +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> > +	if (!irq) {
> > +		dev_err(dev, "failed to get irq\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> > +			       p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to request irq=%d\n", irq);
> > +		return ret;
> > +	}
> > +	dev_dbg(dev, "registered irq=%d\n", irq);
> > +	spin_lock_init(&p1_dev->spinlock_irq);
> > +
> > +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> > +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> > +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> > +	if (!p1_dev->clks)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->num_clks; ++i)
> > +		p1_dev->clks[i].id = clk_names[i];
> > +
> > +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> > +	pm_runtime_use_autosuspend(dev);
> > +	pm_runtime_enable(dev);
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret) {
> > +		isp_teardown_scp_rproc(p1_dev);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> > +	pm_runtime_dont_use_autosuspend(dev);
> > +	pm_runtime_disable(dev);
> > +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> > +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +	isp_teardown_scp_rproc(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > +			   NULL)
> > +};
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam-p1",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> > +MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > new file mode 100644
> > index 000000000000..837662f92a5e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> 
> This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?
> 

Ok, revise in next patch.

> > @@ -0,0 +1,64 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_HW_H__
> > +#define __MTK_CAM_HW_H__
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ipi.h"
> > +
> > +/*
> > + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> > + *
> > + * @dev: Pointer to device.
> > + * @scp_pdev: Pointer to SCP platform device.
> > + * @rproc_handle: Pointer to new remoteproc instance.
> > + * @cam_dev: Embedded struct cam_dev
> > + * @regs: Camera ISP HW base register address
> > + * @num_clks: The number of driver's clocks
> > + * @clks: The clock data array
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> > + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> > + * @composed_frame_seq_no: Frame sequence number of composed frame
> > + * @timestamp: Frame timestamp in ns
> > + * @sof_count: SOF counter
> > + * @composer_wq: The work queue for frame request composing
> > + * @composer_scp_addr: SCP address of ISP composer memory
> > + * @composer_iova: DMA address of ISP composer memory
> > + * @virt_addr: Virtual address of ISP composer memory
> > + *
> > + */
> > +struct mtk_isp_p1_device {
> > +	struct device *dev;
> > +	struct mtk_scp *scp;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_cam_dev cam_dev;
> > +	void __iomem *regs;
> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clks;
> > +	/* Used to protect register read/write data */
> > +	spinlock_t spinlock_irq;
> > +	unsigned int enqueued_frame_seq_no;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int composed_frame_seq_no;
> > +	u8 sof_count;
> > +	struct workqueue_struct *composer_wq;
> > +	dma_addr_t composer_scp_addr;
> > +	dma_addr_t composer_iova;
> > +	void *composer_virt_addr;
> > +};
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> > +		       struct p1_config_param *config_param);
> > +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> > +			 struct mtk_cam_dev_request *req);
> 
> It would be nice to have docs for these too.
> 

Ok, add in next patch.

> > +
> > +#endif /* __MTK_CAM_HW_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > new file mode 100644
> > index 000000000000..981b634dd91f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> 
> I'm skipping this file, since, if I understand correctly, this is not ready for review right?
> 

I think this file is ready for review.


> > @@ -0,0 +1,222 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_IPI_H__
> > +#define __MTK_CAM_IPI_H__
> > +
> > +#include <linux/types.h>
> > +
> > +/*
> > + * struct img_size - Image size information.
> > + *
> > + * @w: Image width, the unit is pixel
> > + * @h: Image height, the unit is pixel
> > + * @xsize: Bytes per line based on width.
> > + * @stride: Bytes per line when changing line.
> > + *          Stride is based on xsize + HW constrain(byte align).
> > + *
> > + */
> > +struct img_size {
> > +	u32 w;
> > +	u32 h;
> > +	u32 xsize;
> > +	u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_crop - image corp information
> > + *
> > + * @left: The left of crop area.
> > + * @top: The top of crop area.
> > + * @width: The width of crop area.
> > + * @height: The height of crop area.
> > + *
> > + */
> > +struct p1_img_crop {
> > +	u32 left;
> > +	u32 top;
> > +	u32 width;
> > +	u32 height;
> > +} __packed;
> > +
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> 
> I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).
> 

Ok, revise in next patch.

> > +	u32 scp_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_output - ISP P1 image output information
> > + *
> > + * @buffer: DMA buffer address of image.
> > + * @size: The image size configuration.
> > + * @crop: The crop configuration.
> > + * @pixel_bits: The bits per image pixel.
> > + * @img_fmt: The image format.
> > + *
> > + */
> > +struct p1_img_output {
> > +	struct dma_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	u8 pixel_bits;
> > +	u32 img_fmt;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_in_param - Image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: Indicate the sensor mode. Continuous or single shot.
> > + * @subsample: Indicate to enables SOF subsample or not.
> > + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: Describe input data pattern.
> > + * @raw_pixel_id: Bayer sequence.
> > + * @tg_fps: The fps rate of TG (time generator).
> > + * @img_fmt: The image format of input source.
> > + * @p1_img_crop: The crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	u8 continuous;
> > +	u8 subsample;
> > +	u8 pixel_mode;
> > +	u8 data_pattern;
> > +	u8 raw_pixel_id;
> > +	u16 tg_fps;
> > +	u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - The image output parameters of main stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @pure_raw: Indicate the image path control.
> > + *            True: pure raw
> > + *            False: processing raw
> > + * @pure_raw_pack: Indicate the image is packed or not.
> > + *                 True: packed mode
> > + *                 False: unpacked mode
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	u8 bypass;
> > +	u8 pure_raw;
> > +	u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - The image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_config_param - ISP P1 configuration parameters.
> > + *
> > + * @cfg_in_param: The Image input parameters.
> > + * @cfg_main_param: The main output image parameters.
> > + * @cfg_resize_out_param: The packed output image parameters.
> > + * @enabled_dmas: The enabled DMA port information.
> > + *
> > + */
> > +struct p1_config_param {
> > +	struct cfg_in_param cfg_in_param;
> > +	struct cfg_main_out_param cfg_main_param;
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	u32 enabled_dmas;
> > +} __packed;
> > +
> > +/*
> > + * struct P1_meta_frame - ISP P1 meta frame information.
> > + *
> > + * @enabled_dma: The enabled DMA port information.
> > + * @vb_index: The VB2 index of meta buffer.
> > + * @meta_addr: DMA buffer address of meta buffer.
> > + *
> > + */
> > +struct P1_meta_frame {
> > +	u32 enabled_dma;
> > +	u32 vb_index;
> > +	struct dma_buffer meta_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_init_info - ISP P1 composer init information.
> > + *
> > + * @hw_module: The ISP Camera HW module ID.
> > + * @cq_addr: The DMA address of composer memory.
> > + *
> > + */
> > +struct isp_init_info {
> > +	u8 hw_module;
> > +	struct dma_buffer cq_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_ack_info - ISP P1 IPI command ack information.
> > + *
> > + * @cmd_id: The IPI command ID is acked.
> > + * @frame_seq_no: The IPI frame sequence number is acked.
> > + *
> > + */
> > +struct isp_ack_info {
> > +	u8 cmd_id;
> > +	u32 frame_seq_no;
> > +} __packed;
> > +
> > +/*
> > + * The IPI command enumeration.
> > + */
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +/*
> > + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> > + *
> > + * @cmd_id: The IPI command ID.
> > + * @init_param: The init formation for ISP_CMD_INIT.
> > + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> > + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> > + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> > + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> > + *
> > + */
> > +struct mtk_isp_scp_p1_cmd {
> > +	u8 cmd_id;
> > +	union {
> > +		struct isp_init_info init_param;
> > +		struct p1_config_param config_param;
> > +		u32 enabled_dmas;
> > +		struct P1_meta_frame meta_frame;
> > +		u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +#endif /* __MTK_CAM_IPI_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..ab2277f45fa4
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,95 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_REGS_H__
> > +#define __MTK_CAM_REGS_H__
> > +
> > +/* ISP interrupt enable */
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define DMA_ERR_INT_EN			BIT(29)
> > +
> > +/* ISP interrupt status */
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define VS_INT_ST			BIT(0)
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define HW_PASS1_DON_ST			BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +#define SW_PASS1_DON_ST			BIT(30)
> > +
> > +/* ISP interrupt 2 status */
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +
> > +/* Configures sensor mode */
> > +#define REG_TG_SEN_MODE			0x0230
> > +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_VF_CON			0x0234
> > +#define TG_VF_CON_VFDATA_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_INTER_ST			0x026c
> > +#define TG_CS_MASK			0x3f00
> > +#define TG_IDLE_ST			BIT(8)
> > +
> > +/* IMGO error status register */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +/* RRZO error status register */
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +/* AAO error status register */
> > +#define REG_AAO_ERR_STAT		0x1368
> > +/* AFO error status register */
> > +#define REG_AFO_ERR_STAT		0x136c
> > +/* LCSO error status register */
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +/* BPCI error status register */
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +/* LSCI error status register */
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +/* LMVO error status register */
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +/* FLKO error status register */
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +/* PSO error status register */
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* CQ0 base address */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +/* Frame sequence number */
> > +#define REG_FRAME_SEQ_NUM		0x13b8
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> 
> I would add a common prefix all the registers in the file.
> 
> Also, add some docs to know what those acronyms means would be nice.
> 

Ok, add this in next patch.


> > +#endif	/* __MTK_CAM_REGS_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..23fdb8b4abc5
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,2087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-dma-contig.h>
> 
> Please sort in alphabetical order.
> 

Fix in next patch

> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_PSO		BIT(10)
> 
> It would be nice to have better names of docs of what these means.
> 

Add in next patch

> > +
> > +#define MTK_ISP_ONE_PIXEL_MODE		1
> > +#define MTK_ISP_MIN_RESIZE_RATIO	6
> > +#define MTK_ISP_MAX_RUNNING_JOBS	3
> > +
> > +#define MTK_CAM_CIO_PAD_SRC		4
> > +#define MTK_CAM_CIO_PAD_SINK		11
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> 
> no need for the underscore in __vq
> 

Revise in next patch

> > +{
> > +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_request *
> > +mtk_cam_req_to_dev_req(struct media_request *__req)
> > +{
> > +	return container_of(__req, struct mtk_cam_dev_request, req);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> > +				 struct mtk_cam_dev_request *req,
> > +				 enum vb2_buffer_state state)
> > +{
> > +	struct media_request_object *obj, *obj_prev;
> > +	unsigned long flags;
> > +	u64 ts_eof = ktime_get_boottime_ns();
> > +
> > +	if (!cam->streaming)
> 
> s/streaming/is_streaming
> 
> this makes a bit more intuitive of what the the boolean means.
> 

Revise in next patch

> > +		return;
> > +
> > +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> > +
> > +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> > +		struct vb2_buffer *vb;
> > +		struct mtk_cam_dev_buffer *buf;
> > +		struct mtk_cam_video_device *node;
> > +
> > +		if (!vb2_request_object_is_buffer(obj))
> > +			continue;
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> > +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +		spin_lock_irqsave(&node->buf_list_lock, flags);
> > +		list_del(&buf->list);
> > +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> > +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > +			vb->timestamp = ts_eof;
> > +		else
> > +			vb->timestamp = req->timestamp;
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +			return req;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +
> > +	return NULL;
> > +}
> > +
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> > +				   unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> > +			list_del(&req->list);
> > +			break;
> > +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> > +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> > +				 req->frame_params.frame_seq_no);
> 
> maybe a counter in debugfs instead of the warning.
> 

Do you mean to add counter to accumulate the total count of drop frames?
Could we add this and also keep this warning message?

> > +			list_del(&req->list);
> > +		} else {
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	if (!cam->streaming) {
> > +		dev_dbg(cam->dev, "stream is off\n");
> > +		return;
> > +	}
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> > +			dev_dbg(cam->dev, "jobs are full\n");
> > +			break;
> > +		}
> > +		cam->running_job_count++;
> > +		list_del(&req->list);
> > +		list_add_tail(&req->list, &cam->running_job_list);
> 
> list_move_tail() can be used.
> 

Revised in this patch.

> > +		mtk_isp_req_enqueue(cam, req);
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> > +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> > +					       media_dev);
> > +	unsigned long flags;
> > +
> > +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> > +	vb2_request_queue(req);
> > +
> > +	/* add to pending job list */
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	mtk_cam_dev_req_try_queue(cam);
> > +}
> > +
> > +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> 
> which patchset are these pixel formats defined?
> I couldn't find them in the ones you pointed.
> 
> I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
> so you can use v4l2_format_info() to get the number of bytes.
> 

I miss some files related to pixel format definition in this patch set.
You could refer the old patch set for pixel format definition.
https://patchwork.kernel.org/patch/11126055/

> > +
> > +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			     struct v4l2_pix_format_mplane *mp)
> > +{
> > +	unsigned int bpl, ppl;
> 
> bytes per line and pixels per line right?
> 

Yes.

> > +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> 
> wouldn't be easier a get_pixel_bytes() function instead of bits?
> 

Sorry. I didn't get the point.
The unit of return value is bits, not bytes.
Do you suggest move bpl & ppl calculation into get_pixel_bits() and
rename to get_pixel_bytes()?

> > +	unsigned int width = mp->width;
> > +
> > +	bpl = 0;
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		/* Bayer encoding format & 2 bytes alignment */
> > +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> > +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		/*
> > +		 * The FULL-G encoding format
> > +		 * 1 G component per pixel
> > +		 * 1 R component per 4 pixel
> > +		 * 1 B component per 4 pixel
> > +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> > +		 */
> > +		ppl = DIV_ROUND_UP(width * 6, 4);
> > +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> > +
> > +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> > +		if (pixel_bits == 10)
> > +			bpl = ALIGN(bpl, 4);
> > +		else
> > +			bpl = ALIGN(bpl, 8);
> > +	}
> > +	/*
> > +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> > +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> > +	 */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	mp->plane_fmt[0].bytesperline = bpl;
> > +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> > +
> > +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> > +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	int i;
> 
> unsigned
> 

Revised in next patch.

> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (!queue_desc->image) {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +		return;
> > +	}
> > +
> > +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> > +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> > +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> > +	dest->fmt.pix_mp.num_planes = 1;
> > +
> > +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_R;
> > +	default:
> > +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> 
> I was wondering if it is not better to save all the media bus format
> into a table, instead of having several swtch case statements.
> 

Ok, revise in next patch.

> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}> +}
> 
> same for the pixelformat.
> 
> Then you can cache object with the pixelformat in the main struct.
> 

Ok, revise in next patch.

> > +
> > +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			  struct p1_img_output *out_fmt, int sd_width,
> > +			  int sd_height)
> > +{
> > +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> > +
> > +	/* Check output & input image size dimension */
> > +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> > +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> > +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> > +			node_id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Check resize ratio for resize out stream due to HW constraint */
> > +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> > +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> > +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> > +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    !out_fmt->pixel_bits) {
> > +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> > +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> > +		return -EINVAL;
> > +	}
> > +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> > +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> > +
> > +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> > +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +	out_fmt->crop.left = 0;
> > +	out_fmt->crop.top = 0;
> > +	out_fmt->crop.width = sd_width;
> > +	out_fmt->crop.height = sd_height;
> > +
> > +	dev_dbg(cam->dev,
> > +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		node_id, out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	cam->enabled_count = 0;
> > +	cam->enabled_dmas = 0;
> > +	cam->stream_count = 0;
> > +	cam->running_job_count = 0;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		if (!cam->vdev_nodes[i].enabled)
> > +			continue;
> > +		cam->enabled_count++;
> > +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> > +	}
> > +
> > +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> > +		cam->enabled_dmas);
> > +}
> > +
> > +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct p1_config_param config_param;
> > +	struct cfg_in_param *cfg_in_param;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	int sd_width, sd_height, sd_code;
> 
> are this sd_* variables required? Can't sd_fmt be directly accessed?
> 

Ok, revised in next patch set.

> > +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> > +	int ret;
> > +
> > +	/* Get sensor format configuration */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> > +		return ret;
> > +	}
> > +	sd_width = sd_fmt.format.width;
> > +	sd_height = sd_fmt.format.height;
> > +	sd_code = sd_fmt.format.code;
> > +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> > +		sd_code);
> 
> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
> it should be configured from userspace.
> 

Could you explain why?
Moreover, how does configuration from user space?

> > +
> > +	memset(&config_param, 0, sizeof(config_param));
> > +
> > +	/* Update cfg_in_param */
> > +	cfg_in_param = &config_param.cfg_in_param;
> > +	cfg_in_param->continuous = true;
> > +	/* Fix to one pixel mode in default */
> > +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> > +	cfg_in_param->crop.width = sd_width;
> > +	cfg_in_param->crop.height = sd_height;
> > +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> > +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> > +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> > +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Update cfg_main_param */
> > +	config_param.cfg_main_param.pure_raw = true;
> > +	config_param.cfg_main_param.pure_raw_pack = true;
> > +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> > +			     &config_param.cfg_main_param.output,
> > +			     sd_width, sd_height);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Update cfg_resize_param */
> > +	if (enabled_dma_ports & R_RRZO) {
> > +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> > +				     &config_param.cfg_resize_param.output,
> > +				     sd_width, sd_height);
> > +		if (ret)
> > +			return ret;
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = true;
> > +	}
> > +
> > +	/* Update enabled_dmas */
> > +	config_param.enabled_dmas = enabled_dma_ports;
> > +	mtk_isp_hw_config(cam, &config_param);
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> > +				  unsigned int frame_seq_no)
> > +{
> > +	struct v4l2_event event = {
> > +		.type = V4L2_EVENT_FRAME_SYNC,
> > +		.u.frame_sync.frame_sequence = frame_seq_no,
> > +	};
> > +
> > +	v4l2_event_queue(cam->subdev.devnode, &event);
> > +}
> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> > +{
> > +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> > +	struct device *dev = cam->dev;
> > +	struct media_entity *entity;
> > +	struct v4l2_subdev *sensor;
> > +
> > +	sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> > +			entity->name, entity->function, entity->stream_count);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "no seninf connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "no seninf connected\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> > +	if (!cam->sensor)
> > +		return -ENODEV;
> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		goto fail_seninf_off;
> > +	}
> > +
> > +	ret = mtk_cam_dev_isp_config(cam);
> > +	if (ret)
> > +		goto fail_sensor_off;
> > +
> > +	cam->streaming = true;
> > +	mtk_isp_stream(cam, 1);
> > +	mtk_cam_dev_req_try_queue(cam);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +fail_seninf_off:
> > +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		return -EPERM;
> 
> Why -EPERM ?
> 

Ok, we will return ret directly.

> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	cam->streaming = false;
> > +	mtk_isp_stream(cam, 0);
> > +	mtk_isp_hw_release(cam);
> > +
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> > +
> > +	if (enable) {
> > +		/* Align vb2_core_streamon design */
> > +		if (cam->streaming) {
> > +			dev_warn(cam->dev, "already streaming on\n");
> 
> I think just dev_dbg is enough.
> 

Fix in next patch.

> > +			return 0;
> > +		}
> > +		return mtk_cam_cio_stream_on(cam);
> > +	}
> > +
> > +	if (!cam->streaming) {
> > +		dev_warn(cam->dev, "already streaming off\n");
> 
> same here
> 

Fix in next patch.

> > +		return 0;
> > +	}
> > +	return mtk_cam_cio_stream_off(cam);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	/*
> > +	 * The video nodes exposed by the driver have pads indexes
> > +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> > +	 */
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> > +		cam->dev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> 
> Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?
> 

Ok, revised in next patch.

> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> 
> isn't it an issue if userspace queue two buffers for the same video device in the same request?
> 
> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
> will be at req->frame_params.dma_bufs[buf->node_id], no?
> 
> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
> buffer?
> 
> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
> 

1. yes, it is a issue if userspace queues two buffers for the same video
device with the same request FD.

2. All buffers which are belonged different to different video devices
in the request list will be updated to req->frame_params.dma_bufs by
buf->node_id.

3. It is not allowed for userspace to queue partial buffers for all
enabled video devices. If it happens, it may trigger DMA errors for this
request.

> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	dma_addr_t addr;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	/* SCP address is only valid for meta input buffer */
> > +	if (!node->desc.smem_alloc)
> > +		return 0;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	/* Use coherent address to get iova address */
> > +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> > +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> > +		return -EFAULT;
> > +	}
> > +	buf->scp_addr = buf->daddr;
> > +	buf->daddr = addr;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size) {
> > +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> > +			vb2_plane_size(vb, 0), size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > +		if (vb2_get_plane_payload(vb, 0) != size) {
> > +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> > +				vb2_get_plane_payload(vb, 0), size);
> > +			return -EINVAL;
> > +		}
> > +		return 0;
> > +	}
> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct device *dev = cam->dev;
> > +
> > +	if (!node->desc.smem_alloc)
> > +		return;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	dma_unmap_page_attrs(dev, buf->daddr,
> > +			     vb->planes[0].length,
> > +			     DMA_BIDIRECTIONAL,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +}
> > +
> > +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (node->desc.smem_alloc)
> > +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size || *num_planes != 1)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> > +		list_del(&buf->list);
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_ret_buf;
> > +	}
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	/* Start streaming of the whole pipeline now*/
> > +	if (!cam->pipeline.streaming_count) {
> 
> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
> 

The check is designed to start the media pipeline when we start
streaming on the first node. You could refer the detail in below link.

https://patchwork.kernel.org/patch/10985819/


> > +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> > +		if (ret) {
> > +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> > +			goto fail_unlock;
> > +		}
> > +		mtk_cam_dev_init_stream(cam);
> > +		ret = mtk_isp_hw_init(cam);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init HW:%d\n", ret);
> > +			goto fail_stop_pipeline;
> > +		}
> > +	}
> > +
> > +	/* Media links are fixed after media_pipeline_start */
> > +	cam->stream_count++;
> > +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> > +		cam->enabled_count);
> > +	if (cam->stream_count < cam->enabled_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return 0;
> > +	}
> > +
> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> > +	if (ret)
> > +		goto fail_no_stream;
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	return 0;
> > +
> > +fail_no_stream:
> > +	cam->stream_count--;
> > +fail_stop_pipeline:
> > +	if (cam->stream_count == 0)
> > +		media_pipeline_stop(&node->vdev.entity);
> > +fail_unlock:
> > +	mutex_unlock(&cam->op_lock);
> > +fail_ret_buf:
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> > +		cam->stream_count);
> > +	/* Check the first node to stream-off */
> > +	if (cam->stream_count == cam->enabled_count)
> > +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> > +
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> > +	cam->stream_count--;
> > +	if (cam->stream_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return;
> > +	}
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	mtk_cam_dev_req_cleanup(cam);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> > +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam->dev));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct device *dev = cam->dev;
> > +	const struct v4l2_format *dev_fmt;
> > +	struct v4l2_format try_fmt;
> > +
> > +	memset(&try_fmt, 0, sizeof(try_fmt));
> > +	try_fmt.type = f->type;
> > +
> > +	/* Validate pixelformat */
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> > +	if (!dev_fmt) {
> > +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> > +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> > +	}
> > +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> > +
> > +	/* Validate image width & height range */
> > +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> > +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> > +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> > +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> > +	/* 4 bytes alignment for width */
> > +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> > +
> > +	/* Only support one plane */
> > +	try_fmt.fmt.pix_mp.num_planes = 1;
> > +
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> > +
> > +	/* Constant format fields */
> > +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +
> > +	*f = try_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (vb2_is_busy(node->vdev.queue)) {
> > +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> > +		return -EBUSY;
> > +	}
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +	/* Configure to video device */
> > +	node->vdev_fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> 
> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
> I wonder if pad ops shouldn't be implemented too (to be verified).
> 

Ok, I will investigate this.

> > +
> > +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> > +	.buf_request_complete = mtk_cam_vb2_request_complete,
> > +};> +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_ops = {
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> > +				  struct media_device *media_dev)
> > +{
> > +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> > +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	media_dev->dev = cam->dev;
> > +	strscpy(media_dev->model, dev_driver_string(dev),
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_ops;
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize subdev pads */
> > +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> > +					sizeof(*cam->subdev_pads),
> > +					GFP_KERNEL);
> > +	if (!cam->subdev_pads) {
> > +		dev_err(dev, "failed to allocate subdev_pads\n");
> > +		ret = -ENOMEM;
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> > +				     cam->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_pads; i++)
> > +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	return 0;
> > +
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> > +			      struct mtk_cam_video_device *node)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> > +	unsigned int link_flags = node->desc.link_flags;
> > +	int ret;
> > +
> > +	/* Initialize mtk_cam_video_device */
> > +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> > +		node->enabled = true;
> > +	else
> > +		node->enabled = false;
> > +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> > +
> > +	cam->subdev_pads[node->id].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->desc.buf_type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam->smem_dev;
> > +	} else {
> > +		vbq->dev = dev;
> > +	}
> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> > +	if (output)
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> > +	else
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> > +	/* No minimum buffers limitation */
> > +	vbq->min_buffers_needed = 0;
> > +	vbq->drv_priv = cam;
> > +	vbq->lock = &node->vdev_lock;
> > +	vbq->supports_requests = true;
> > +	vbq->requires_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_media_clean;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 dev_driver_string(dev), node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->vdev_lock;
> > +	vdev->v4l2_dev = &cam->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> > +	vdev->entity.ops = NULL;
> > +	video_set_drvdata(vdev, cam);
> > +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->vdev_lock);
> > +	INIT_LIST_HEAD(&node->buf_list);
> > +	spin_lock_init(&node->buf_list_lock);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vb2_rel;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam->subdev.entity,
> > +					    node->id, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam->subdev.entity,
> > +					    node->id, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> 
> No need for the curly braces.
> 

Revised in next patch.

> > +	if (ret)
> > +		goto fail_vdev_ureg;
> > +
> > +	return 0;
> > +
> > +fail_vdev_ureg:
> > +	video_unregister_device(vdev);
> > +fail_vb2_rel:
> > +	mutex_destroy(&node->vdev_lock);
> > +	vb2_queue_release(vbq);
> > +fail_media_clean:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static void
> > +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> > +{
> > +	video_unregister_device(&node->vdev);
> > +	vb2_queue_release(&node->vbq);
> > +	media_entity_cleanup(&node->vdev.entity);
> > +	mutex_destroy(&node->vdev_lock);
> > +}
> > +
> > +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	/* Set up media device & pads */
> > +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> > +	if (ret)
> > +		return ret;
> > +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> > +
> > +	/* Set up v4l2 device */
> > +	cam->v4l2_dev.mdev = &cam->media_dev;
> > +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> > +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> > +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> > +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> > +		 "%s", dev_driver_string(dev));
> > +	v4l2_set_subdevdata(&cam->subdev, cam);
> > +
> > +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> > +		goto fail_clean_media_entiy;
> > +	}
> > +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> > +
> > +		node->id = node->desc.id;
> > +		ret = mtk_cam_video_register_device(cam, node);
> > +		if (ret)
> > +			goto fail_vdev_unreg;
> > +	}
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_vdev_unreg:
> > +	for (i--; i >= 0; i--)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +fail_clean_media_entiy:
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +
> > +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> > +	v4l2_device_unregister_subdev(&cam->subdev);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> > +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	cam->seninf = sd;
> > +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam->seninf = NULL;
> > +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "No seninf subdev\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> > +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> > +				    MEDIA_LNK_FL_IMMUTABLE |
> > +				    MEDIA_LNK_FL_ENABLED);
> > +	if (ret) {
> > +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> > +			cam->seninf->entity.name, cam->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	v4l2_async_notifier_init(&cam->notifier);
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> 
> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
> 
> Regards,
> Helen
> 

Ok, we will investigate how to do.

> > +	if (ret) {
> > +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> > +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	v4l2_async_notifier_unregister(&cam->notifier);
> > +	v4l2_async_notifier_cleanup(&cam->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};> +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 512 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 1200 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 640 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 288 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	/* This is a default image format */
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> > +{
> > +	unsigned int node_idx;
> > +	int i;
> > +
> > +	node_idx = 0;
> > +	/* Setup the output queue */
> > +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam)
> > +{
> > +	int ret;
> > +
> > +	cam->dev = &pdev->dev;
> > +	mtk_cam_dev_queue_setup(cam);
> > +
> > +	spin_lock_init(&cam->pending_job_lock);
> > +	spin_lock_init(&cam->running_job_lock);
> > +	INIT_LIST_HEAD(&cam->pending_job_list);
> > +	INIT_LIST_HEAD(&cam->running_job_list);
> > +	mutex_init(&cam->op_lock);
> > +
> > +	/* v4l2 sub-device registration */
> > +	ret = mtk_cam_v4l2_register(cam);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam);
> > +	if (ret)
> > +		goto fail_v4l2_unreg;
> > +
> > +	return 0;
> > +
> > +fail_v4l2_unreg:
> > +	mutex_destroy(&cam->op_lock);
> > +	mtk_cam_v4l2_unregister(cam);
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam);
> > +	mtk_cam_v4l2_unregister(cam);
> > +	mutex_destroy(&cam->op_lock);
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..0a340a1e65ea
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,244 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_H__
> > +#define __MTK_CAM_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "mtk_cam-ipi.h"
> > +
> > +#define IMG_MAX_WIDTH		5376
> > +#define IMG_MAX_HEIGHT		4032
> > +#define IMG_MIN_WIDTH		80
> > +#define IMG_MIN_HEIGHT		60
> > +
> > +/*
> > + * ID enum value for struct mtk_cam_dev_node_desc:id
> > + * or mtk_cam_video_device:id
> > + */
> > +enum  {
> > +	MTK_CAM_P1_META_IN_0 = 0,
> > +	MTK_CAM_P1_MAIN_STREAM_OUT,
> > +	MTK_CAM_P1_PACKED_BIN_OUT,
> > +	MTK_CAM_P1_META_OUT_0,
> > +	MTK_CAM_P1_META_OUT_1,
> > +	MTK_CAM_P1_META_OUT_2,
> > +	MTK_CAM_P1_META_OUT_3,
> > +	MTK_CAM_P1_TOTAL_NODES
> > +};
> > +
> > +/* Supported image format list */
> > +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> > +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> > +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> > +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> > +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> > +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> > +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> > +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> > +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> > +
> > +/* Supported bayer pixel order */
> > +#define MTK_CAM_RAW_PXL_ID_B		0
> > +#define MTK_CAM_RAW_PXL_ID_GB		1
> > +#define MTK_CAM_RAW_PXL_ID_GR		2
> > +#define MTK_CAM_RAW_PXL_ID_R		3
> > +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> > +
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +/*
> > + * struct mtk_cam_dev_request - MTK camera device request.
> > + *
> > + * @req: Embedded struct media request.
> > + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> > + * @frame_work: work queue entry for frame transmission to SCP.
> > + * @list: List entry of the object for @struct mtk_cam_dev:
> > + *        pending_job_list or running_job_list.
> > + * @timestamp: Start of frame timestamp in ns
> > + *
> > + */
> > +struct mtk_cam_dev_request {
> > +	struct media_request req;
> > +	struct mtk_p1_frame_param frame_params;
> > +	struct work_struct frame_work;
> > +	struct list_head list;
> > +	u64 timestamp;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> > + *
> > + * @vbb: Embedded struct vb2_v4l2_buffer.
> > + * @list: List entry of the object for @struct mtk_cam_video_device:
> > + *        buf_list.
> > + * @daddr: The DMA address of this buffer.
> > + * @scp_addr: The SCP address of this buffer which
> > + *            is only supported for meta input node.
> > + * @node_id: The vidoe node id which this buffer belongs to.
> > + *
> > + */
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer vbb;
> > +	struct list_head list;
> > +	/* Intenal part */
> > +	dma_addr_t daddr;
> > +	dma_addr_t scp_addr;
> > +	unsigned int node_id;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> > + *
> > + * @id: id of the node
> > + * @name: name of the node
> > + * @cap: supported V4L2 capabilities
> > + * @buf_type: supported V4L2 buffer type
> > + * @dma_port: the dma ports associated to the node
> > + * @link_flags: default media link flags
> > + * @smem_alloc: using the smem_dev as alloc device or not
> > + * @image: true for image node, false for meta node
> > + * @num_fmts: the number of supported node formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum VB2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts: supported format
> > + * @frmsizes: supported V4L2 frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	const char *name;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure
> > + *
> > + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> > + * @enabled: Indicate the video device is enabled or not
> > + * @desc: The node description of video device
> > + * @vdev_fmt: The V4L2 format of video device
> > + * @vdev_pad: The media pad graph object of video device
> > + * @vbq: A videobuf queue of video device
> > + * @vdev: The video device instance
> > + * @vdev_lock: Serializes vb2 queue and video device operations
> > + * @buf_list: List for enqueue buffers
> > + * @buf_list_lock: Lock used to protect buffer list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_format vdev_fmt;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct video_device vdev;
> > +	/* Serializes vb2 queue and video device operations */
> > +	struct mutex vdev_lock;
> > +	struct list_head buf_list;
> > +	/* Lock used to protect buffer list */
> > +	spinlock_t buf_list_lock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @dev: Pointer to device.
> > + * @smem_pdev: Pointer to shared memory device.
> > + * @pipeline: Media pipeline information.
> > + * @media_dev: Media device instance.
> > + * @subdev: The V4L2 sub-device instance.
> > + * @v4l2_dev: The V4L2 device driver instance.
> > + * @notifier: The v4l2_device notifier data.
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> > + * @seninf: Pointer to the seninf sub-device.
> > + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> > + * @streaming: Indicate the overall streaming status is on or off.
> > + * @enabled_dmas: The enabled dma port information when streaming on.
> > + * @enabled_count: Number of enabled video nodes
> > + * @stream_count: Number of streaming video nodes
> > + * @running_job_count: Nunber of running jobs in the HW driver.
> > + * @pending_job_list: List to keep the media requests before en-queue into
> > + *                    HW driver.
> > + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> > + * @running_job_list: List to keep the media requests after en-queue into
> > + *                    HW driver.
> > + * @running_job_lock: Protect the running_job_list data.
> > + * @op_lock: Serializes driver's VB2 callback operations.
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct device *dev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int enabled_dmas;
> > +	unsigned int enabled_count;
> > +	unsigned int stream_count;
> > +	unsigned int running_job_count;
> > +	struct list_head pending_job_list;
> > +	/* Protect the pending_job_list data */
> > +	spinlock_t pending_job_lock;
> > +	struct list_head running_job_list;
> > +	/* Protect the running_job_list data & running_job_count */
> > +	spinlock_t running_job_lock;
> > +	/* Serializes driver's VB2 callback operations */
> > +	struct mutex op_lock;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> > +				   unsigned int frame_seq_no);
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				  unsigned int frame_seq_no);
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no);
> > +
> > +#endif /* __MTK_CAM_H__ */
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-04-09  2:05         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-09  2:05 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport

Hi Helen:

Thanks for your comments.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hello Jungo,
> 
> I was taking a look at this patch (thanks for the work),
> I didn't look in deep details, but I have some comments, please see
> below. I hope it helps.
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >  - Correct auto suspend timer value for suspend/resume issue
> >  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >  - Fix KE due to no sen-inf sub-device
> > ---
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> 
> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
> capture, which for metadata, and what is being used to deal with requests.
> 
> It would make it easier to review imho.
> 

For file structure design, it was reviewed in the previous patch
serials.
e.g.
https://patchwork.kernel.org/patch/10938137/
If you think it is better, I will modify it.

> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> 
> It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.
> 
> Or maybe something like:
> 
> mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
> mtkisp_p1_capture.c
> mtkisp_p1_meta.c
> mtkisp_p1_isp.c
> mtkisp_p1_hw.c (or maybe split this between the other files)
> mtkisp_p1_request.c
> mtkisp_p1_common.c (?)
> 
> or s/mtkisp_p1/mtk_cam/
> 
> what do you think?
> 

Ok, I will revise our naming issue for consistency reason.

> >  9 files changed, 3377 insertions(+)
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> > new file mode 100644
> > index 000000000000..f86e1b59ad1e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/Kconfig
> > @@ -0,0 +1,20 @@
> > +config VIDEO_MEDIATEK_ISP_PASS1
> > +	tristate "Mediatek ISP Pass 1 driver"
> > +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> 
> I think you need OF as well
> 
> depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF
> 
> > +	depends on ARCH_MEDIATEK
> 
> depends on ARCH_MEDIATEK || COMPILE_TEST
> 

Ok, we will fix this in next patch.

> > +	select V4L2_FWNODE
> > +	select VIDEOBUF2_VMALLOC
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select MTK_SCP
> > +	default n
> > +	help
> > +		Pass 1 driver controls 3A (auto-focus, exposure,
> > +		and white balance) with tuning feature and outputs
> > +		the captured image buffers in Mediatek's camera system.
> > +
> > +		Choose Y if you want to use Mediatek SoCs to create image
> > +		captured application such as video recording and still image
> > +		capturing.
> 
> I would re-word this a bit, since people can use a captured application (and not create one) :)
> 

Ok, I will re-word as "if you want to use image captured application
based on Mediatek SoCs for video recording and still image capturing
functions"

> > +
> > +		To compile this driver as a module, choose M here; the module
> > +		will be called mtk-cam-isp.
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > new file mode 100644
> > index 000000000000..ce79d283b209
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..53b54d3c26a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +mtk-cam-isp-objs += mtk_cam.o
> > +mtk-cam-isp-objs += mtk_cam-hw.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > new file mode 100644
> > index 000000000000..4065d0d29b7f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > @@ -0,0 +1,636 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/module.h>
> > +#include <linux/remoteproc/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include <media/v4l2-event.h>
> 
> Please sort headers alphabetically.
> 

Will fix in next patch.

> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +#include "mtk_cam-regs.h"
> > +
> > +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> > +#define MTK_ISP_CQ_BUFFER_COUNT			3
> > +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> > +
> > +/*
> > + *
> > + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> > + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> > + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> > + * In current driver, only supports CAM B.
> > + *
> > + */
> > +#define MTK_ISP_CAM_ID_B			3
> > +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> > +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> > +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> > +
> > +static void isp_tx_frame_worker(struct work_struct *work)
> 
> I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
> an external function.
> 

Fix in next patch.

> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > +
> > +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> > +		dev_err(dev, "wrong IPI len:%d\n", len);
> > +		return;
> > +	}
> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> > +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> > +		return;
> > +
> > +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> > +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> > +}
> > +
> > +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI cmd\n");
> > +		return ret;
> > +	}
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI frame\n");
> > +		goto unreg_ipi_cmd;
> > +	}
> > +
> > +	p1_dev->composer_wq =
> > +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> > +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> > +					WQ_FREEZABLE);
> > +	if (!p1_dev->composer_wq) {
> > +		dev_err(dev, "failed to alloc composer workqueue\n");
> > +		goto unreg_ipi_frame;
> > +	}
> > +
> > +	return 0;
> > +
> > +unreg_ipi_frame:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +unreg_ipi_cmd:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	destroy_workqueue(p1_dev->composer_wq);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +}
> > +
> > +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> > +
> > +	/*
> > +	 * Passed coherent reserved memory info. for SCP firmware usage.
> > +	 * This buffer is used for SCP's ISP composer to compose.
> > +	 * The size of is fixed to 0x200000 for the requirement of composer.
> > +	 */
> > +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> > +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +
> > +	isp_composer_uninit(p1_dev);
> 
> I think you can copy the 3 lines of this isp_composer_uninit() function here, since
> this seems the only place it is being used, and having a deinit and uninit function is
> a bit confusing.
> 

Fix in next patch.

> > +}
> > +
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> > +		       struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> 
> I prefer not having a int parameter, this is easier to read:
> 
> mtk_isp_stream_on(cam);
> mtk_isp_stream_off(cam);
> 
> or
> 
> mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
> mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);
> 
> instead of:
> 
> mtk_isp_stream(cam, 1);
> mtk_isp_stream(cam, 0);
> 
> You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.
> 

Ok, I will choose the method 2.

> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> 
> s/is_stream_on/is_streaming
> 

Fix in next patch.

> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		dev_err(dev, "failed to rproc_boot\n");
> 
> It would be nice to improve this error message for users, how about:
> 
> dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);
> 
> Or maybe even remove this message, since rproc_boot() already have several error messages.
> 

Ok, we will remove the error message.

> > +		return ret;
> > +	}
> > +
> > +	ret = isp_composer_init(p1_dev);
> > +	if (ret)
> 
> should rproc_shutdown() be called here?
> 

Yes, we will fix it.

> > +		return ret;
> > +
> > +	pm_runtime_get_sync(dev);
> 
> You should check return value here.
> 

Fix in next patch.

> > +	isp_composer_hw_init(p1_dev);
> > +
> > +	p1_dev->enqueued_frame_seq_no = 0;
> > +	p1_dev->dequeued_frame_seq_no = 0;
> > +	p1_dev->composed_frame_seq_no = 0;
> > +	p1_dev->sof_count = 0;
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	isp_composer_hw_deinit(p1_dev);
> > +	pm_runtime_mark_last_busy(dev);
> > +	pm_runtime_put_autosuspend(dev);
> > +	rproc_shutdown(p1_dev->rproc_handle);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> > +
> > +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> > +			       unsigned int dequeued_frame_seq_no)
> > +{
> > +	dma_addr_t base_addr = p1_dev->composer_iova;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_cam_dev_request *req;
> > +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> > +	unsigned int addr_offset;
> > +
> > +	/* Send V4L2_EVENT_FRAME_SYNC event */
> > +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +
> > +	p1_dev->sof_count += 1;
> > +	/* Save frame information */
> > +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> > +
> > +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +	if (req)
> > +		req->timestamp = ktime_get_boottime_ns();
> > +
> > +	/* Update CQ base address if needed */
> > +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> > +		dev_dbg(dev,
> > +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> > +			composed_frame_seq_no, dequeued_frame_seq_no);
> > +		return;
> > +	}
> > +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> > +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> > +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> > +}
> > +
> > +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	u32 val;
> > +
> > +	dev_err(p1_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(p1_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> 
> I think if would be better to transfor those into dev_dbg and add a counter
> in debugfs.
> 

These error messages are important for debugging.
I suggest to keep in dev_err.

Moreover, could you give more information about debug counter?
I don't get your point.
Do you suggest to accumulate the total count of DMA errors?

> > +
> > +	/* Disable DMA error mask to avoid too much error log */
> > +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> > +}
> > +
> > +static irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> > +	struct device *dev = p1_dev->dev;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> > +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> > +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> > +
> > +	/*
> > +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> > +	 * If these two ISRs come together, print warning msg to hint.
> > +	 */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> > +
> > +	/* De-queue frame */
> > +	if (irq_status & SW_PASS1_DON_ST) {
> 
> I suppose this means "done streaming"?
> 

Yes, it means the frame buffer is outputed done.

> > +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> > +					      p1_dev->dequeued_frame_seq_no);
> > +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> > +	}
> > +
> > +	/* Save frame info. & update CQ address for frame HW en-queue */
> > +	if (irq_status & SOF_INT_ST)
> > +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_irq_handle_dma_err(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> > +		p1_dev->sof_count, irq_status, dma_status,
> > +		dequeued_frame_seq_no);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> > +			       struct platform_device *pdev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	dma_addr_t addr;
> > +	void *ptr;
> 
> Maybe "composer_buffer" would be a better name.
> 
> But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?
> 

Ok, I will use p1_dev->composer_virt_addr directly.

> > +	int ret;
> > +
> > +	p1_dev->scp = scp_get(pdev);
> > +	if (!p1_dev->scp) {
> > +		dev_err(dev, "failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> > +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> 
> I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
> which mapping we are refering to.
> 

Fix in next patch.

> > +
> > +	/*
> > +	 * Allocate coherent reserved memory for SCP firmware usage.
> > +	 * The size of SCP composer's memory is fixed to 0x200000
> > +	 * for the requirement of firmware.
> > +	 */
> > +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> > +	if (!ptr) {
> > +		ret = -ENOMEM;
> > +		goto fail_put_scp;
> > +	}
> > +
> > +	p1_dev->composer_scp_addr = addr;
> > +	p1_dev->composer_virt_addr = ptr;
> > +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> > +
> > +	/*
> > +	 * This reserved memory is also be used by ISP P1 HW.
> > +	 * Need to get iova address for ISP P1 DMA.
> > +	 */
> > +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> > +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map scp iova\n");
> > +		ret = -ENOMEM;
> > +		goto fail_free_mem;
> > +	}
> > +	p1_dev->composer_iova = addr;
> 
> why not rename this to composer_isp_addr ?
> Since, afaik, composer_scp_addr is also iova.
> 
> At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).
> 

Ok, we will rename composer_iova to composer_isp_addr.
Basically, scp_addr is reserved physical address and it is not behind an
IOMMU.

> > +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> > +
> > +	return 0;
> > +
> > +fail_free_mem:
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +fail_put_scp:
> > +	scp_put(p1_dev->scp);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +	scp_put(p1_dev->scp);
> > +}
> > +
> > +static int mtk_isp_pm_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Disable ISP's view finder and wait for TG idle if possible */
> > +	dev_dbg(dev, "cam suspend, disable VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> > +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> > +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> > +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> > +
> > +	/* Disable CMOS */
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Force ISP HW to idle */
> > +	ret = pm_runtime_force_suspend(dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to force suspend:%d\n", ret);
> > +		goto reenable_hw;
> > +	}
> > +
> > +	return 0;
> > +
> > +reenable_hw:
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_pm_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Force ISP HW to resume */
> > +	ret = pm_runtime_force_resume(dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Enable CMOS */
> > +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Enable VF */
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	dev_dbg(dev, "%s:disable clock\n", __func__);
> > +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s:enable clock\n", __func__);
> > +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to enable clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	/* List of clocks required by isp cam */
> > +	static const char * const clk_names[] = {
> > +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +	};
> > +	struct mtk_isp_p1_device *p1_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq, ret, i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->dev = dev;
> > +	dev_set_drvdata(dev, p1_dev);
> > +
> > +	/*
> > +	 * Now only support single CAM with CAM B.
> > +	 * Get CAM B register base with CAM B index.
> > +	 * Support multiple CAMs in future.
> > +	 */
> > +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> > +	p1_dev->regs = devm_ioremap_resource(dev, res);
> > +	if (IS_ERR(p1_dev->regs)) {
> > +		dev_err(dev, "failed to map reister base\n");
> 
> s/reister/register
> 

Fix in next patch.

> > +		return PTR_ERR(p1_dev->regs);
> > +	}
> > +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> > +
> > +	/*
> > +	 * The cam_sys unit only supports reg., but has no IRQ support.
> > +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> > +	 */
> > +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> > +	if (!irq) {
> > +		dev_err(dev, "failed to get irq\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> > +			       p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to request irq=%d\n", irq);
> > +		return ret;
> > +	}
> > +	dev_dbg(dev, "registered irq=%d\n", irq);
> > +	spin_lock_init(&p1_dev->spinlock_irq);
> > +
> > +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> > +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> > +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> > +	if (!p1_dev->clks)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->num_clks; ++i)
> > +		p1_dev->clks[i].id = clk_names[i];
> > +
> > +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> > +	pm_runtime_use_autosuspend(dev);
> > +	pm_runtime_enable(dev);
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret) {
> > +		isp_teardown_scp_rproc(p1_dev);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> > +	pm_runtime_dont_use_autosuspend(dev);
> > +	pm_runtime_disable(dev);
> > +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> > +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +	isp_teardown_scp_rproc(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > +			   NULL)
> > +};
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam-p1",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> > +MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > new file mode 100644
> > index 000000000000..837662f92a5e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> 
> This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?
> 

Ok, revise in next patch.

> > @@ -0,0 +1,64 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_HW_H__
> > +#define __MTK_CAM_HW_H__
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ipi.h"
> > +
> > +/*
> > + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> > + *
> > + * @dev: Pointer to device.
> > + * @scp_pdev: Pointer to SCP platform device.
> > + * @rproc_handle: Pointer to new remoteproc instance.
> > + * @cam_dev: Embedded struct cam_dev
> > + * @regs: Camera ISP HW base register address
> > + * @num_clks: The number of driver's clocks
> > + * @clks: The clock data array
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> > + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> > + * @composed_frame_seq_no: Frame sequence number of composed frame
> > + * @timestamp: Frame timestamp in ns
> > + * @sof_count: SOF counter
> > + * @composer_wq: The work queue for frame request composing
> > + * @composer_scp_addr: SCP address of ISP composer memory
> > + * @composer_iova: DMA address of ISP composer memory
> > + * @virt_addr: Virtual address of ISP composer memory
> > + *
> > + */
> > +struct mtk_isp_p1_device {
> > +	struct device *dev;
> > +	struct mtk_scp *scp;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_cam_dev cam_dev;
> > +	void __iomem *regs;
> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clks;
> > +	/* Used to protect register read/write data */
> > +	spinlock_t spinlock_irq;
> > +	unsigned int enqueued_frame_seq_no;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int composed_frame_seq_no;
> > +	u8 sof_count;
> > +	struct workqueue_struct *composer_wq;
> > +	dma_addr_t composer_scp_addr;
> > +	dma_addr_t composer_iova;
> > +	void *composer_virt_addr;
> > +};
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> > +		       struct p1_config_param *config_param);
> > +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> > +			 struct mtk_cam_dev_request *req);
> 
> It would be nice to have docs for these too.
> 

Ok, add in next patch.

> > +
> > +#endif /* __MTK_CAM_HW_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > new file mode 100644
> > index 000000000000..981b634dd91f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> 
> I'm skipping this file, since, if I understand correctly, this is not ready for review right?
> 

I think this file is ready for review.


> > @@ -0,0 +1,222 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_IPI_H__
> > +#define __MTK_CAM_IPI_H__
> > +
> > +#include <linux/types.h>
> > +
> > +/*
> > + * struct img_size - Image size information.
> > + *
> > + * @w: Image width, the unit is pixel
> > + * @h: Image height, the unit is pixel
> > + * @xsize: Bytes per line based on width.
> > + * @stride: Bytes per line when changing line.
> > + *          Stride is based on xsize + HW constrain(byte align).
> > + *
> > + */
> > +struct img_size {
> > +	u32 w;
> > +	u32 h;
> > +	u32 xsize;
> > +	u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_crop - image corp information
> > + *
> > + * @left: The left of crop area.
> > + * @top: The top of crop area.
> > + * @width: The width of crop area.
> > + * @height: The height of crop area.
> > + *
> > + */
> > +struct p1_img_crop {
> > +	u32 left;
> > +	u32 top;
> > +	u32 width;
> > +	u32 height;
> > +} __packed;
> > +
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> 
> I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).
> 

Ok, revise in next patch.

> > +	u32 scp_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_output - ISP P1 image output information
> > + *
> > + * @buffer: DMA buffer address of image.
> > + * @size: The image size configuration.
> > + * @crop: The crop configuration.
> > + * @pixel_bits: The bits per image pixel.
> > + * @img_fmt: The image format.
> > + *
> > + */
> > +struct p1_img_output {
> > +	struct dma_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	u8 pixel_bits;
> > +	u32 img_fmt;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_in_param - Image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: Indicate the sensor mode. Continuous or single shot.
> > + * @subsample: Indicate to enables SOF subsample or not.
> > + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: Describe input data pattern.
> > + * @raw_pixel_id: Bayer sequence.
> > + * @tg_fps: The fps rate of TG (time generator).
> > + * @img_fmt: The image format of input source.
> > + * @p1_img_crop: The crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	u8 continuous;
> > +	u8 subsample;
> > +	u8 pixel_mode;
> > +	u8 data_pattern;
> > +	u8 raw_pixel_id;
> > +	u16 tg_fps;
> > +	u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - The image output parameters of main stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @pure_raw: Indicate the image path control.
> > + *            True: pure raw
> > + *            False: processing raw
> > + * @pure_raw_pack: Indicate the image is packed or not.
> > + *                 True: packed mode
> > + *                 False: unpacked mode
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	u8 bypass;
> > +	u8 pure_raw;
> > +	u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - The image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_config_param - ISP P1 configuration parameters.
> > + *
> > + * @cfg_in_param: The Image input parameters.
> > + * @cfg_main_param: The main output image parameters.
> > + * @cfg_resize_out_param: The packed output image parameters.
> > + * @enabled_dmas: The enabled DMA port information.
> > + *
> > + */
> > +struct p1_config_param {
> > +	struct cfg_in_param cfg_in_param;
> > +	struct cfg_main_out_param cfg_main_param;
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	u32 enabled_dmas;
> > +} __packed;
> > +
> > +/*
> > + * struct P1_meta_frame - ISP P1 meta frame information.
> > + *
> > + * @enabled_dma: The enabled DMA port information.
> > + * @vb_index: The VB2 index of meta buffer.
> > + * @meta_addr: DMA buffer address of meta buffer.
> > + *
> > + */
> > +struct P1_meta_frame {
> > +	u32 enabled_dma;
> > +	u32 vb_index;
> > +	struct dma_buffer meta_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_init_info - ISP P1 composer init information.
> > + *
> > + * @hw_module: The ISP Camera HW module ID.
> > + * @cq_addr: The DMA address of composer memory.
> > + *
> > + */
> > +struct isp_init_info {
> > +	u8 hw_module;
> > +	struct dma_buffer cq_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_ack_info - ISP P1 IPI command ack information.
> > + *
> > + * @cmd_id: The IPI command ID is acked.
> > + * @frame_seq_no: The IPI frame sequence number is acked.
> > + *
> > + */
> > +struct isp_ack_info {
> > +	u8 cmd_id;
> > +	u32 frame_seq_no;
> > +} __packed;
> > +
> > +/*
> > + * The IPI command enumeration.
> > + */
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +/*
> > + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> > + *
> > + * @cmd_id: The IPI command ID.
> > + * @init_param: The init formation for ISP_CMD_INIT.
> > + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> > + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> > + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> > + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> > + *
> > + */
> > +struct mtk_isp_scp_p1_cmd {
> > +	u8 cmd_id;
> > +	union {
> > +		struct isp_init_info init_param;
> > +		struct p1_config_param config_param;
> > +		u32 enabled_dmas;
> > +		struct P1_meta_frame meta_frame;
> > +		u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +#endif /* __MTK_CAM_IPI_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..ab2277f45fa4
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,95 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_REGS_H__
> > +#define __MTK_CAM_REGS_H__
> > +
> > +/* ISP interrupt enable */
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define DMA_ERR_INT_EN			BIT(29)
> > +
> > +/* ISP interrupt status */
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define VS_INT_ST			BIT(0)
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define HW_PASS1_DON_ST			BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +#define SW_PASS1_DON_ST			BIT(30)
> > +
> > +/* ISP interrupt 2 status */
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +
> > +/* Configures sensor mode */
> > +#define REG_TG_SEN_MODE			0x0230
> > +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_VF_CON			0x0234
> > +#define TG_VF_CON_VFDATA_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_INTER_ST			0x026c
> > +#define TG_CS_MASK			0x3f00
> > +#define TG_IDLE_ST			BIT(8)
> > +
> > +/* IMGO error status register */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +/* RRZO error status register */
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +/* AAO error status register */
> > +#define REG_AAO_ERR_STAT		0x1368
> > +/* AFO error status register */
> > +#define REG_AFO_ERR_STAT		0x136c
> > +/* LCSO error status register */
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +/* BPCI error status register */
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +/* LSCI error status register */
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +/* LMVO error status register */
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +/* FLKO error status register */
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +/* PSO error status register */
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* CQ0 base address */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +/* Frame sequence number */
> > +#define REG_FRAME_SEQ_NUM		0x13b8
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> 
> I would add a common prefix all the registers in the file.
> 
> Also, add some docs to know what those acronyms means would be nice.
> 

Ok, add this in next patch.


> > +#endif	/* __MTK_CAM_REGS_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..23fdb8b4abc5
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,2087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-dma-contig.h>
> 
> Please sort in alphabetical order.
> 

Fix in next patch

> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_PSO		BIT(10)
> 
> It would be nice to have better names of docs of what these means.
> 

Add in next patch

> > +
> > +#define MTK_ISP_ONE_PIXEL_MODE		1
> > +#define MTK_ISP_MIN_RESIZE_RATIO	6
> > +#define MTK_ISP_MAX_RUNNING_JOBS	3
> > +
> > +#define MTK_CAM_CIO_PAD_SRC		4
> > +#define MTK_CAM_CIO_PAD_SINK		11
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> 
> no need for the underscore in __vq
> 

Revise in next patch

> > +{
> > +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_request *
> > +mtk_cam_req_to_dev_req(struct media_request *__req)
> > +{
> > +	return container_of(__req, struct mtk_cam_dev_request, req);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> > +				 struct mtk_cam_dev_request *req,
> > +				 enum vb2_buffer_state state)
> > +{
> > +	struct media_request_object *obj, *obj_prev;
> > +	unsigned long flags;
> > +	u64 ts_eof = ktime_get_boottime_ns();
> > +
> > +	if (!cam->streaming)
> 
> s/streaming/is_streaming
> 
> this makes a bit more intuitive of what the the boolean means.
> 

Revise in next patch

> > +		return;
> > +
> > +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> > +
> > +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> > +		struct vb2_buffer *vb;
> > +		struct mtk_cam_dev_buffer *buf;
> > +		struct mtk_cam_video_device *node;
> > +
> > +		if (!vb2_request_object_is_buffer(obj))
> > +			continue;
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> > +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +		spin_lock_irqsave(&node->buf_list_lock, flags);
> > +		list_del(&buf->list);
> > +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> > +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > +			vb->timestamp = ts_eof;
> > +		else
> > +			vb->timestamp = req->timestamp;
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +			return req;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +
> > +	return NULL;
> > +}
> > +
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> > +				   unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> > +			list_del(&req->list);
> > +			break;
> > +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> > +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> > +				 req->frame_params.frame_seq_no);
> 
> maybe a counter in debugfs instead of the warning.
> 

Do you mean to add counter to accumulate the total count of drop frames?
Could we add this and also keep this warning message?

> > +			list_del(&req->list);
> > +		} else {
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	if (!cam->streaming) {
> > +		dev_dbg(cam->dev, "stream is off\n");
> > +		return;
> > +	}
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> > +			dev_dbg(cam->dev, "jobs are full\n");
> > +			break;
> > +		}
> > +		cam->running_job_count++;
> > +		list_del(&req->list);
> > +		list_add_tail(&req->list, &cam->running_job_list);
> 
> list_move_tail() can be used.
> 

Revised in this patch.

> > +		mtk_isp_req_enqueue(cam, req);
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> > +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> > +					       media_dev);
> > +	unsigned long flags;
> > +
> > +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> > +	vb2_request_queue(req);
> > +
> > +	/* add to pending job list */
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	mtk_cam_dev_req_try_queue(cam);
> > +}
> > +
> > +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> 
> which patchset are these pixel formats defined?
> I couldn't find them in the ones you pointed.
> 
> I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
> so you can use v4l2_format_info() to get the number of bytes.
> 

I miss some files related to pixel format definition in this patch set.
You could refer the old patch set for pixel format definition.
https://patchwork.kernel.org/patch/11126055/

> > +
> > +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			     struct v4l2_pix_format_mplane *mp)
> > +{
> > +	unsigned int bpl, ppl;
> 
> bytes per line and pixels per line right?
> 

Yes.

> > +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> 
> wouldn't be easier a get_pixel_bytes() function instead of bits?
> 

Sorry. I didn't get the point.
The unit of return value is bits, not bytes.
Do you suggest move bpl & ppl calculation into get_pixel_bits() and
rename to get_pixel_bytes()?

> > +	unsigned int width = mp->width;
> > +
> > +	bpl = 0;
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		/* Bayer encoding format & 2 bytes alignment */
> > +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> > +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		/*
> > +		 * The FULL-G encoding format
> > +		 * 1 G component per pixel
> > +		 * 1 R component per 4 pixel
> > +		 * 1 B component per 4 pixel
> > +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> > +		 */
> > +		ppl = DIV_ROUND_UP(width * 6, 4);
> > +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> > +
> > +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> > +		if (pixel_bits == 10)
> > +			bpl = ALIGN(bpl, 4);
> > +		else
> > +			bpl = ALIGN(bpl, 8);
> > +	}
> > +	/*
> > +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> > +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> > +	 */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	mp->plane_fmt[0].bytesperline = bpl;
> > +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> > +
> > +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> > +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	int i;
> 
> unsigned
> 

Revised in next patch.

> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (!queue_desc->image) {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +		return;
> > +	}
> > +
> > +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> > +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> > +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> > +	dest->fmt.pix_mp.num_planes = 1;
> > +
> > +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_R;
> > +	default:
> > +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> 
> I was wondering if it is not better to save all the media bus format
> into a table, instead of having several swtch case statements.
> 

Ok, revise in next patch.

> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}> +}
> 
> same for the pixelformat.
> 
> Then you can cache object with the pixelformat in the main struct.
> 

Ok, revise in next patch.

> > +
> > +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			  struct p1_img_output *out_fmt, int sd_width,
> > +			  int sd_height)
> > +{
> > +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> > +
> > +	/* Check output & input image size dimension */
> > +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> > +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> > +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> > +			node_id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Check resize ratio for resize out stream due to HW constraint */
> > +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> > +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> > +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> > +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    !out_fmt->pixel_bits) {
> > +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> > +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> > +		return -EINVAL;
> > +	}
> > +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> > +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> > +
> > +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> > +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +	out_fmt->crop.left = 0;
> > +	out_fmt->crop.top = 0;
> > +	out_fmt->crop.width = sd_width;
> > +	out_fmt->crop.height = sd_height;
> > +
> > +	dev_dbg(cam->dev,
> > +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		node_id, out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	cam->enabled_count = 0;
> > +	cam->enabled_dmas = 0;
> > +	cam->stream_count = 0;
> > +	cam->running_job_count = 0;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		if (!cam->vdev_nodes[i].enabled)
> > +			continue;
> > +		cam->enabled_count++;
> > +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> > +	}
> > +
> > +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> > +		cam->enabled_dmas);
> > +}
> > +
> > +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct p1_config_param config_param;
> > +	struct cfg_in_param *cfg_in_param;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	int sd_width, sd_height, sd_code;
> 
> are this sd_* variables required? Can't sd_fmt be directly accessed?
> 

Ok, revised in next patch set.

> > +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> > +	int ret;
> > +
> > +	/* Get sensor format configuration */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> > +		return ret;
> > +	}
> > +	sd_width = sd_fmt.format.width;
> > +	sd_height = sd_fmt.format.height;
> > +	sd_code = sd_fmt.format.code;
> > +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> > +		sd_code);
> 
> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
> it should be configured from userspace.
> 

Could you explain why?
Moreover, how does configuration from user space?

> > +
> > +	memset(&config_param, 0, sizeof(config_param));
> > +
> > +	/* Update cfg_in_param */
> > +	cfg_in_param = &config_param.cfg_in_param;
> > +	cfg_in_param->continuous = true;
> > +	/* Fix to one pixel mode in default */
> > +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> > +	cfg_in_param->crop.width = sd_width;
> > +	cfg_in_param->crop.height = sd_height;
> > +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> > +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> > +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> > +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Update cfg_main_param */
> > +	config_param.cfg_main_param.pure_raw = true;
> > +	config_param.cfg_main_param.pure_raw_pack = true;
> > +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> > +			     &config_param.cfg_main_param.output,
> > +			     sd_width, sd_height);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Update cfg_resize_param */
> > +	if (enabled_dma_ports & R_RRZO) {
> > +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> > +				     &config_param.cfg_resize_param.output,
> > +				     sd_width, sd_height);
> > +		if (ret)
> > +			return ret;
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = true;
> > +	}
> > +
> > +	/* Update enabled_dmas */
> > +	config_param.enabled_dmas = enabled_dma_ports;
> > +	mtk_isp_hw_config(cam, &config_param);
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> > +				  unsigned int frame_seq_no)
> > +{
> > +	struct v4l2_event event = {
> > +		.type = V4L2_EVENT_FRAME_SYNC,
> > +		.u.frame_sync.frame_sequence = frame_seq_no,
> > +	};
> > +
> > +	v4l2_event_queue(cam->subdev.devnode, &event);
> > +}
> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> > +{
> > +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> > +	struct device *dev = cam->dev;
> > +	struct media_entity *entity;
> > +	struct v4l2_subdev *sensor;
> > +
> > +	sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> > +			entity->name, entity->function, entity->stream_count);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "no seninf connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "no seninf connected\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> > +	if (!cam->sensor)
> > +		return -ENODEV;
> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		goto fail_seninf_off;
> > +	}
> > +
> > +	ret = mtk_cam_dev_isp_config(cam);
> > +	if (ret)
> > +		goto fail_sensor_off;
> > +
> > +	cam->streaming = true;
> > +	mtk_isp_stream(cam, 1);
> > +	mtk_cam_dev_req_try_queue(cam);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +fail_seninf_off:
> > +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		return -EPERM;
> 
> Why -EPERM ?
> 

Ok, we will return ret directly.

> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	cam->streaming = false;
> > +	mtk_isp_stream(cam, 0);
> > +	mtk_isp_hw_release(cam);
> > +
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> > +
> > +	if (enable) {
> > +		/* Align vb2_core_streamon design */
> > +		if (cam->streaming) {
> > +			dev_warn(cam->dev, "already streaming on\n");
> 
> I think just dev_dbg is enough.
> 

Fix in next patch.

> > +			return 0;
> > +		}
> > +		return mtk_cam_cio_stream_on(cam);
> > +	}
> > +
> > +	if (!cam->streaming) {
> > +		dev_warn(cam->dev, "already streaming off\n");
> 
> same here
> 

Fix in next patch.

> > +		return 0;
> > +	}
> > +	return mtk_cam_cio_stream_off(cam);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	/*
> > +	 * The video nodes exposed by the driver have pads indexes
> > +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> > +	 */
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> > +		cam->dev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> 
> Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?
> 

Ok, revised in next patch.

> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> 
> isn't it an issue if userspace queue two buffers for the same video device in the same request?
> 
> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
> will be at req->frame_params.dma_bufs[buf->node_id], no?
> 
> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
> buffer?
> 
> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
> 

1. yes, it is a issue if userspace queues two buffers for the same video
device with the same request FD.

2. All buffers which are belonged different to different video devices
in the request list will be updated to req->frame_params.dma_bufs by
buf->node_id.

3. It is not allowed for userspace to queue partial buffers for all
enabled video devices. If it happens, it may trigger DMA errors for this
request.

> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	dma_addr_t addr;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	/* SCP address is only valid for meta input buffer */
> > +	if (!node->desc.smem_alloc)
> > +		return 0;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	/* Use coherent address to get iova address */
> > +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> > +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> > +		return -EFAULT;
> > +	}
> > +	buf->scp_addr = buf->daddr;
> > +	buf->daddr = addr;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size) {
> > +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> > +			vb2_plane_size(vb, 0), size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > +		if (vb2_get_plane_payload(vb, 0) != size) {
> > +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> > +				vb2_get_plane_payload(vb, 0), size);
> > +			return -EINVAL;
> > +		}
> > +		return 0;
> > +	}
> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct device *dev = cam->dev;
> > +
> > +	if (!node->desc.smem_alloc)
> > +		return;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	dma_unmap_page_attrs(dev, buf->daddr,
> > +			     vb->planes[0].length,
> > +			     DMA_BIDIRECTIONAL,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +}
> > +
> > +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (node->desc.smem_alloc)
> > +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size || *num_planes != 1)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> > +		list_del(&buf->list);
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_ret_buf;
> > +	}
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	/* Start streaming of the whole pipeline now*/
> > +	if (!cam->pipeline.streaming_count) {
> 
> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
> 

The check is designed to start the media pipeline when we start
streaming on the first node. You could refer the detail in below link.

https://patchwork.kernel.org/patch/10985819/


> > +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> > +		if (ret) {
> > +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> > +			goto fail_unlock;
> > +		}
> > +		mtk_cam_dev_init_stream(cam);
> > +		ret = mtk_isp_hw_init(cam);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init HW:%d\n", ret);
> > +			goto fail_stop_pipeline;
> > +		}
> > +	}
> > +
> > +	/* Media links are fixed after media_pipeline_start */
> > +	cam->stream_count++;
> > +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> > +		cam->enabled_count);
> > +	if (cam->stream_count < cam->enabled_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return 0;
> > +	}
> > +
> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> > +	if (ret)
> > +		goto fail_no_stream;
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	return 0;
> > +
> > +fail_no_stream:
> > +	cam->stream_count--;
> > +fail_stop_pipeline:
> > +	if (cam->stream_count == 0)
> > +		media_pipeline_stop(&node->vdev.entity);
> > +fail_unlock:
> > +	mutex_unlock(&cam->op_lock);
> > +fail_ret_buf:
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> > +		cam->stream_count);
> > +	/* Check the first node to stream-off */
> > +	if (cam->stream_count == cam->enabled_count)
> > +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> > +
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> > +	cam->stream_count--;
> > +	if (cam->stream_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return;
> > +	}
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	mtk_cam_dev_req_cleanup(cam);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> > +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam->dev));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct device *dev = cam->dev;
> > +	const struct v4l2_format *dev_fmt;
> > +	struct v4l2_format try_fmt;
> > +
> > +	memset(&try_fmt, 0, sizeof(try_fmt));
> > +	try_fmt.type = f->type;
> > +
> > +	/* Validate pixelformat */
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> > +	if (!dev_fmt) {
> > +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> > +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> > +	}
> > +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> > +
> > +	/* Validate image width & height range */
> > +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> > +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> > +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> > +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> > +	/* 4 bytes alignment for width */
> > +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> > +
> > +	/* Only support one plane */
> > +	try_fmt.fmt.pix_mp.num_planes = 1;
> > +
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> > +
> > +	/* Constant format fields */
> > +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +
> > +	*f = try_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (vb2_is_busy(node->vdev.queue)) {
> > +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> > +		return -EBUSY;
> > +	}
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +	/* Configure to video device */
> > +	node->vdev_fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> 
> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
> I wonder if pad ops shouldn't be implemented too (to be verified).
> 

Ok, I will investigate this.

> > +
> > +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> > +	.buf_request_complete = mtk_cam_vb2_request_complete,
> > +};> +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_ops = {
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> > +				  struct media_device *media_dev)
> > +{
> > +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> > +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	media_dev->dev = cam->dev;
> > +	strscpy(media_dev->model, dev_driver_string(dev),
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_ops;
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize subdev pads */
> > +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> > +					sizeof(*cam->subdev_pads),
> > +					GFP_KERNEL);
> > +	if (!cam->subdev_pads) {
> > +		dev_err(dev, "failed to allocate subdev_pads\n");
> > +		ret = -ENOMEM;
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> > +				     cam->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_pads; i++)
> > +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	return 0;
> > +
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> > +			      struct mtk_cam_video_device *node)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> > +	unsigned int link_flags = node->desc.link_flags;
> > +	int ret;
> > +
> > +	/* Initialize mtk_cam_video_device */
> > +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> > +		node->enabled = true;
> > +	else
> > +		node->enabled = false;
> > +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> > +
> > +	cam->subdev_pads[node->id].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->desc.buf_type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam->smem_dev;
> > +	} else {
> > +		vbq->dev = dev;
> > +	}
> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> > +	if (output)
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> > +	else
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> > +	/* No minimum buffers limitation */
> > +	vbq->min_buffers_needed = 0;
> > +	vbq->drv_priv = cam;
> > +	vbq->lock = &node->vdev_lock;
> > +	vbq->supports_requests = true;
> > +	vbq->requires_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_media_clean;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 dev_driver_string(dev), node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->vdev_lock;
> > +	vdev->v4l2_dev = &cam->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> > +	vdev->entity.ops = NULL;
> > +	video_set_drvdata(vdev, cam);
> > +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->vdev_lock);
> > +	INIT_LIST_HEAD(&node->buf_list);
> > +	spin_lock_init(&node->buf_list_lock);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vb2_rel;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam->subdev.entity,
> > +					    node->id, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam->subdev.entity,
> > +					    node->id, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> 
> No need for the curly braces.
> 

Revised in next patch.

> > +	if (ret)
> > +		goto fail_vdev_ureg;
> > +
> > +	return 0;
> > +
> > +fail_vdev_ureg:
> > +	video_unregister_device(vdev);
> > +fail_vb2_rel:
> > +	mutex_destroy(&node->vdev_lock);
> > +	vb2_queue_release(vbq);
> > +fail_media_clean:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static void
> > +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> > +{
> > +	video_unregister_device(&node->vdev);
> > +	vb2_queue_release(&node->vbq);
> > +	media_entity_cleanup(&node->vdev.entity);
> > +	mutex_destroy(&node->vdev_lock);
> > +}
> > +
> > +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	/* Set up media device & pads */
> > +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> > +	if (ret)
> > +		return ret;
> > +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> > +
> > +	/* Set up v4l2 device */
> > +	cam->v4l2_dev.mdev = &cam->media_dev;
> > +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> > +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> > +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> > +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> > +		 "%s", dev_driver_string(dev));
> > +	v4l2_set_subdevdata(&cam->subdev, cam);
> > +
> > +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> > +		goto fail_clean_media_entiy;
> > +	}
> > +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> > +
> > +		node->id = node->desc.id;
> > +		ret = mtk_cam_video_register_device(cam, node);
> > +		if (ret)
> > +			goto fail_vdev_unreg;
> > +	}
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_vdev_unreg:
> > +	for (i--; i >= 0; i--)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +fail_clean_media_entiy:
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +
> > +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> > +	v4l2_device_unregister_subdev(&cam->subdev);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> > +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	cam->seninf = sd;
> > +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam->seninf = NULL;
> > +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "No seninf subdev\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> > +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> > +				    MEDIA_LNK_FL_IMMUTABLE |
> > +				    MEDIA_LNK_FL_ENABLED);
> > +	if (ret) {
> > +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> > +			cam->seninf->entity.name, cam->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	v4l2_async_notifier_init(&cam->notifier);
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> 
> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
> 
> Regards,
> Helen
> 

Ok, we will investigate how to do.

> > +	if (ret) {
> > +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> > +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	v4l2_async_notifier_unregister(&cam->notifier);
> > +	v4l2_async_notifier_cleanup(&cam->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};> +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 512 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 1200 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 640 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 288 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	/* This is a default image format */
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> > +{
> > +	unsigned int node_idx;
> > +	int i;
> > +
> > +	node_idx = 0;
> > +	/* Setup the output queue */
> > +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam)
> > +{
> > +	int ret;
> > +
> > +	cam->dev = &pdev->dev;
> > +	mtk_cam_dev_queue_setup(cam);
> > +
> > +	spin_lock_init(&cam->pending_job_lock);
> > +	spin_lock_init(&cam->running_job_lock);
> > +	INIT_LIST_HEAD(&cam->pending_job_list);
> > +	INIT_LIST_HEAD(&cam->running_job_list);
> > +	mutex_init(&cam->op_lock);
> > +
> > +	/* v4l2 sub-device registration */
> > +	ret = mtk_cam_v4l2_register(cam);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam);
> > +	if (ret)
> > +		goto fail_v4l2_unreg;
> > +
> > +	return 0;
> > +
> > +fail_v4l2_unreg:
> > +	mutex_destroy(&cam->op_lock);
> > +	mtk_cam_v4l2_unregister(cam);
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam);
> > +	mtk_cam_v4l2_unregister(cam);
> > +	mutex_destroy(&cam->op_lock);
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..0a340a1e65ea
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,244 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_H__
> > +#define __MTK_CAM_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "mtk_cam-ipi.h"
> > +
> > +#define IMG_MAX_WIDTH		5376
> > +#define IMG_MAX_HEIGHT		4032
> > +#define IMG_MIN_WIDTH		80
> > +#define IMG_MIN_HEIGHT		60
> > +
> > +/*
> > + * ID enum value for struct mtk_cam_dev_node_desc:id
> > + * or mtk_cam_video_device:id
> > + */
> > +enum  {
> > +	MTK_CAM_P1_META_IN_0 = 0,
> > +	MTK_CAM_P1_MAIN_STREAM_OUT,
> > +	MTK_CAM_P1_PACKED_BIN_OUT,
> > +	MTK_CAM_P1_META_OUT_0,
> > +	MTK_CAM_P1_META_OUT_1,
> > +	MTK_CAM_P1_META_OUT_2,
> > +	MTK_CAM_P1_META_OUT_3,
> > +	MTK_CAM_P1_TOTAL_NODES
> > +};
> > +
> > +/* Supported image format list */
> > +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> > +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> > +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> > +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> > +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> > +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> > +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> > +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> > +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> > +
> > +/* Supported bayer pixel order */
> > +#define MTK_CAM_RAW_PXL_ID_B		0
> > +#define MTK_CAM_RAW_PXL_ID_GB		1
> > +#define MTK_CAM_RAW_PXL_ID_GR		2
> > +#define MTK_CAM_RAW_PXL_ID_R		3
> > +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> > +
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +/*
> > + * struct mtk_cam_dev_request - MTK camera device request.
> > + *
> > + * @req: Embedded struct media request.
> > + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> > + * @frame_work: work queue entry for frame transmission to SCP.
> > + * @list: List entry of the object for @struct mtk_cam_dev:
> > + *        pending_job_list or running_job_list.
> > + * @timestamp: Start of frame timestamp in ns
> > + *
> > + */
> > +struct mtk_cam_dev_request {
> > +	struct media_request req;
> > +	struct mtk_p1_frame_param frame_params;
> > +	struct work_struct frame_work;
> > +	struct list_head list;
> > +	u64 timestamp;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> > + *
> > + * @vbb: Embedded struct vb2_v4l2_buffer.
> > + * @list: List entry of the object for @struct mtk_cam_video_device:
> > + *        buf_list.
> > + * @daddr: The DMA address of this buffer.
> > + * @scp_addr: The SCP address of this buffer which
> > + *            is only supported for meta input node.
> > + * @node_id: The vidoe node id which this buffer belongs to.
> > + *
> > + */
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer vbb;
> > +	struct list_head list;
> > +	/* Intenal part */
> > +	dma_addr_t daddr;
> > +	dma_addr_t scp_addr;
> > +	unsigned int node_id;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> > + *
> > + * @id: id of the node
> > + * @name: name of the node
> > + * @cap: supported V4L2 capabilities
> > + * @buf_type: supported V4L2 buffer type
> > + * @dma_port: the dma ports associated to the node
> > + * @link_flags: default media link flags
> > + * @smem_alloc: using the smem_dev as alloc device or not
> > + * @image: true for image node, false for meta node
> > + * @num_fmts: the number of supported node formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum VB2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts: supported format
> > + * @frmsizes: supported V4L2 frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	const char *name;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure
> > + *
> > + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> > + * @enabled: Indicate the video device is enabled or not
> > + * @desc: The node description of video device
> > + * @vdev_fmt: The V4L2 format of video device
> > + * @vdev_pad: The media pad graph object of video device
> > + * @vbq: A videobuf queue of video device
> > + * @vdev: The video device instance
> > + * @vdev_lock: Serializes vb2 queue and video device operations
> > + * @buf_list: List for enqueue buffers
> > + * @buf_list_lock: Lock used to protect buffer list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_format vdev_fmt;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct video_device vdev;
> > +	/* Serializes vb2 queue and video device operations */
> > +	struct mutex vdev_lock;
> > +	struct list_head buf_list;
> > +	/* Lock used to protect buffer list */
> > +	spinlock_t buf_list_lock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @dev: Pointer to device.
> > + * @smem_pdev: Pointer to shared memory device.
> > + * @pipeline: Media pipeline information.
> > + * @media_dev: Media device instance.
> > + * @subdev: The V4L2 sub-device instance.
> > + * @v4l2_dev: The V4L2 device driver instance.
> > + * @notifier: The v4l2_device notifier data.
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> > + * @seninf: Pointer to the seninf sub-device.
> > + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> > + * @streaming: Indicate the overall streaming status is on or off.
> > + * @enabled_dmas: The enabled dma port information when streaming on.
> > + * @enabled_count: Number of enabled video nodes
> > + * @stream_count: Number of streaming video nodes
> > + * @running_job_count: Nunber of running jobs in the HW driver.
> > + * @pending_job_list: List to keep the media requests before en-queue into
> > + *                    HW driver.
> > + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> > + * @running_job_list: List to keep the media requests after en-queue into
> > + *                    HW driver.
> > + * @running_job_lock: Protect the running_job_list data.
> > + * @op_lock: Serializes driver's VB2 callback operations.
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct device *dev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int enabled_dmas;
> > +	unsigned int enabled_count;
> > +	unsigned int stream_count;
> > +	unsigned int running_job_count;
> > +	struct list_head pending_job_list;
> > +	/* Protect the pending_job_list data */
> > +	spinlock_t pending_job_lock;
> > +	struct list_head running_job_list;
> > +	/* Protect the running_job_list data & running_job_count */
> > +	spinlock_t running_job_lock;
> > +	/* Serializes driver's VB2 callback operations */
> > +	struct mutex op_lock;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> > +				   unsigned int frame_seq_no);
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				  unsigned int frame_seq_no);
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no);
> > +
> > +#endif /* __MTK_CAM_H__ */
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2020-04-02 16:45       ` Dafna Hirschfeld
@ 2020-04-09  2:49         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-09  2:49 UTC (permalink / raw)
  To: Dafna Hirschfeld
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media

Hi, Dafna:

Thanks for your comments.

On Thu, 2020-04-02 at 18:45 +0200, Dafna Hirschfeld wrote:
> 
> On 19.12.19 06:49, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >   - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >   - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >   - Correct auto suspend timer value for suspend/resume issue
> >   - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >   - Fix KE due to no sen-inf sub-device
> > ---
> >   drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >   .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >   .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >   .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >   9 files changed, 3377 insertions(+)
> >   create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> > new file mode 100644
> > index 000000000000..f86e1b59ad1e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/Kconfig
> > @@ -0,0 +1,20 @@
> > +config VIDEO_MEDIATEK_ISP_PASS1
> > +	tristate "Mediatek ISP Pass 1 driver"
> > +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> > +	depends on ARCH_MEDIATEK
> > +	select V4L2_FWNODE
> > +	select VIDEOBUF2_VMALLOC
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select MTK_SCP
> > +	default n
> > +	help
> > +		Pass 1 driver controls 3A (auto-focus, exposure,
> > +		and white balance) with tuning feature and outputs
> > +		the captured image buffers in Mediatek's camera system.
> > +
> > +		Choose Y if you want to use Mediatek SoCs to create image
> > +		captured application such as video recording and still image
> > +		capturing.
> > +
> > +		To compile this driver as a module, choose M here; the module
> > +		will be called mtk-cam-isp.
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > new file mode 100644
> > index 000000000000..ce79d283b209
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..53b54d3c26a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +mtk-cam-isp-objs += mtk_cam.o
> > +mtk-cam-isp-objs += mtk_cam-hw.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > new file mode 100644
> > index 000000000000..4065d0d29b7f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > @@ -0,0 +1,636 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/module.h>
> > +#include <linux/remoteproc/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include <media/v4l2-event.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +#include "mtk_cam-regs.h"
> > +
> > +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> > +#define MTK_ISP_CQ_BUFFER_COUNT			3
> > +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> > +
> > +/*
> > + *
> > + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> > + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> > + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> > + * In current driver, only supports CAM B.
> > + *
> > + */
> > +#define MTK_ISP_CAM_ID_B			3
> > +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> > +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> > +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> > +
> > +static void isp_tx_frame_worker(struct work_struct *work)
> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > +
> > +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> > +		dev_err(dev, "wrong IPI len:%d\n", len);
> > +		return;
> > +	}
> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> > +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> > +		return;
> > +
> > +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> > +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> > +}
> > +
> > +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI cmd\n");
> > +		return ret;
> > +	}
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI frame\n");
> > +		goto unreg_ipi_cmd;
> > +	}
> > +
> > +	p1_dev->composer_wq =
> > +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> > +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> > +					WQ_FREEZABLE);
> > +	if (!p1_dev->composer_wq) {
> > +		dev_err(dev, "failed to alloc composer workqueue\n");
> > +		goto unreg_ipi_frame;
> > +	}
> > +
> > +	return 0;
> > +
> > +unreg_ipi_frame:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +unreg_ipi_cmd:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	destroy_workqueue(p1_dev->composer_wq);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +}
> > +
> > +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> > +
> > +	/*
> > +	 * Passed coherent reserved memory info. for SCP firmware usage.
> > +	 * This buffer is used for SCP's ISP composer to compose.
> > +	 * The size of is fixed to 0x200000 for the requirement of composer.
> > +	 */
> > +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> > +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +
> > +	isp_composer_uninit(p1_dev);
> > +}
> > +
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> > +		       struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		dev_err(dev, "failed to rproc_boot\n");
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_composer_init(p1_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_get_sync(dev);
> > +	isp_composer_hw_init(p1_dev);
> > +
> > +	p1_dev->enqueued_frame_seq_no = 0;
> > +	p1_dev->dequeued_frame_seq_no = 0;
> > +	p1_dev->composed_frame_seq_no = 0;
> > +	p1_dev->sof_count = 0;
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	isp_composer_hw_deinit(p1_dev);
> > +	pm_runtime_mark_last_busy(dev);
> > +	pm_runtime_put_autosuspend(dev);
> > +	rproc_shutdown(p1_dev->rproc_handle);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> > +
> > +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> > +			       unsigned int dequeued_frame_seq_no)
> > +{
> > +	dma_addr_t base_addr = p1_dev->composer_iova;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_cam_dev_request *req;
> > +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> > +	unsigned int addr_offset;
> > +
> > +	/* Send V4L2_EVENT_FRAME_SYNC event */
> > +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +
> > +	p1_dev->sof_count += 1;
> > +	/* Save frame information */
> > +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> > +
> > +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +	if (req)
> > +		req->timestamp = ktime_get_boottime_ns();
> > +
> > +	/* Update CQ base address if needed */
> > +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> > +		dev_dbg(dev,
> > +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> > +			composed_frame_seq_no, dequeued_frame_seq_no);
> > +		return;
> > +	}
> > +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> > +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> > +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> > +}
> > +
> > +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	u32 val;
> > +
> > +	dev_err(p1_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(p1_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> > +
> > +	/* Disable DMA error mask to avoid too much error log */
> > +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> > +}
> > +
> > +static irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> > +	struct device *dev = p1_dev->dev;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> > +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> > +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> > +
> > +	/*
> > +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> > +	 * If these two ISRs come together, print warning msg to hint.
> > +	 */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> > +
> > +	/* De-queue frame */
> > +	if (irq_status & SW_PASS1_DON_ST) {
> > +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> > +					      p1_dev->dequeued_frame_seq_no);
> > +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> > +	}
> > +
> > +	/* Save frame info. & update CQ address for frame HW en-queue */
> > +	if (irq_status & SOF_INT_ST)
> > +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_irq_handle_dma_err(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> > +		p1_dev->sof_count, irq_status, dma_status,
> > +		dequeued_frame_seq_no);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> > +			       struct platform_device *pdev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	dma_addr_t addr;
> > +	void *ptr;
> > +	int ret;
> > +
> > +	p1_dev->scp = scp_get(pdev);
> > +	if (!p1_dev->scp) {
> > +		dev_err(dev, "failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> > +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> > +
> > +	/*
> > +	 * Allocate coherent reserved memory for SCP firmware usage.
> > +	 * The size of SCP composer's memory is fixed to 0x200000
> > +	 * for the requirement of firmware.
> > +	 */
> > +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> > +	if (!ptr) {
> > +		ret = -ENOMEM;
> > +		goto fail_put_scp;
> > +	}
> > +
> > +	p1_dev->composer_scp_addr = addr;
> > +	p1_dev->composer_virt_addr = ptr;
> > +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> > +
> > +	/*
> > +	 * This reserved memory is also be used by ISP P1 HW.
> > +	 * Need to get iova address for ISP P1 DMA.
> > +	 */
> > +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> > +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map scp iova\n");
> > +		ret = -ENOMEM;
> > +		goto fail_free_mem;
> > +	}
> > +	p1_dev->composer_iova = addr;
> > +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> > +
> > +	return 0;
> > +
> > +fail_free_mem:
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +fail_put_scp:
> > +	scp_put(p1_dev->scp);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +	scp_put(p1_dev->scp);
> > +}
> > +
> > +static int mtk_isp_pm_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Disable ISP's view finder and wait for TG idle if possible */
> > +	dev_dbg(dev, "cam suspend, disable VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> > +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> > +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> > +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> > +
> > +	/* Disable CMOS */
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Force ISP HW to idle */
> > +	ret = pm_runtime_force_suspend(dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to force suspend:%d\n", ret);
> > +		goto reenable_hw;
> > +	}
> > +
> > +	return 0;
> > +
> > +reenable_hw:
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_pm_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Force ISP HW to resume */
> > +	ret = pm_runtime_force_resume(dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Enable CMOS */
> > +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Enable VF */
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	dev_dbg(dev, "%s:disable clock\n", __func__);
> > +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s:enable clock\n", __func__);
> > +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to enable clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	/* List of clocks required by isp cam */
> > +	static const char * const clk_names[] = {
> > +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +	};
> > +	struct mtk_isp_p1_device *p1_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq, ret, i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->dev = dev;
> > +	dev_set_drvdata(dev, p1_dev);
> > +
> > +	/*
> > +	 * Now only support single CAM with CAM B.
> > +	 * Get CAM B register base with CAM B index.
> > +	 * Support multiple CAMs in future.
> > +	 */
> > +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> > +	p1_dev->regs = devm_ioremap_resource(dev, res);
> > +	if (IS_ERR(p1_dev->regs)) {
> > +		dev_err(dev, "failed to map reister base\n");
> > +		return PTR_ERR(p1_dev->regs);
> > +	}
> > +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> > +
> > +	/*
> > +	 * The cam_sys unit only supports reg., but has no IRQ support.
> > +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> > +	 */
> > +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> > +	if (!irq) {
> > +		dev_err(dev, "failed to get irq\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> > +			       p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to request irq=%d\n", irq);
> > +		return ret;
> > +	}
> > +	dev_dbg(dev, "registered irq=%d\n", irq);
> > +	spin_lock_init(&p1_dev->spinlock_irq);
> > +
> > +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> > +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> > +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> > +	if (!p1_dev->clks)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->num_clks; ++i)
> > +		p1_dev->clks[i].id = clk_names[i];
> > +
> > +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> > +	pm_runtime_use_autosuspend(dev);
> > +	pm_runtime_enable(dev);
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret) {
> > +		isp_teardown_scp_rproc(p1_dev);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> > +	pm_runtime_dont_use_autosuspend(dev);
> > +	pm_runtime_disable(dev);
> > +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> > +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +	isp_teardown_scp_rproc(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > +			   NULL)
> > +};
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam-p1",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> > +MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > new file mode 100644
> > index 000000000000..837662f92a5e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > @@ -0,0 +1,64 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_HW_H__
> > +#define __MTK_CAM_HW_H__
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ipi.h"
> > +
> > +/*
> > + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> > + *
> > + * @dev: Pointer to device.
> > + * @scp_pdev: Pointer to SCP platform device.
> > + * @rproc_handle: Pointer to new remoteproc instance.
> > + * @cam_dev: Embedded struct cam_dev
> > + * @regs: Camera ISP HW base register address
> > + * @num_clks: The number of driver's clocks
> > + * @clks: The clock data array
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> > + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> > + * @composed_frame_seq_no: Frame sequence number of composed frame
> > + * @timestamp: Frame timestamp in ns
> > + * @sof_count: SOF counter
> > + * @composer_wq: The work queue for frame request composing
> > + * @composer_scp_addr: SCP address of ISP composer memory
> > + * @composer_iova: DMA address of ISP composer memory
> > + * @virt_addr: Virtual address of ISP composer memory
> > + *
> > + */
> > +struct mtk_isp_p1_device {
> > +	struct device *dev;
> > +	struct mtk_scp *scp;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_cam_dev cam_dev;
> > +	void __iomem *regs;
> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clks;
> > +	/* Used to protect register read/write data */
> > +	spinlock_t spinlock_irq;
> > +	unsigned int enqueued_frame_seq_no;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int composed_frame_seq_no;
> > +	u8 sof_count;
> > +	struct workqueue_struct *composer_wq;
> > +	dma_addr_t composer_scp_addr;
> > +	dma_addr_t composer_iova;
> > +	void *composer_virt_addr;
> > +};
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> > +		       struct p1_config_param *config_param);
> > +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> > +			 struct mtk_cam_dev_request *req);
> > +
> > +#endif /* __MTK_CAM_HW_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > new file mode 100644
> > index 000000000000..981b634dd91f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > @@ -0,0 +1,222 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_IPI_H__
> > +#define __MTK_CAM_IPI_H__
> > +
> > +#include <linux/types.h>
> > +
> > +/*
> > + * struct img_size - Image size information.
> > + *
> > + * @w: Image width, the unit is pixel
> > + * @h: Image height, the unit is pixel
> > + * @xsize: Bytes per line based on width.
> > + * @stride: Bytes per line when changing line.
> > + *          Stride is based on xsize + HW constrain(byte align).
> > + *
> > + */
> > +struct img_size {
> > +	u32 w;
> > +	u32 h;
> > +	u32 xsize;
> > +	u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_crop - image corp information
> > + *
> > + * @left: The left of crop area.
> > + * @top: The top of crop area.
> > + * @width: The width of crop area.
> > + * @height: The height of crop area.
> > + *
> > + */
> > +struct p1_img_crop {
> > +	u32 left;
> > +	u32 top;
> > +	u32 width;
> > +	u32 height;
> > +} __packed;
> > +
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> > +	u32 scp_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_output - ISP P1 image output information
> > + *
> > + * @buffer: DMA buffer address of image.
> > + * @size: The image size configuration.
> > + * @crop: The crop configuration.
> > + * @pixel_bits: The bits per image pixel.
> > + * @img_fmt: The image format.
> > + *
> > + */
> > +struct p1_img_output {
> > +	struct dma_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	u8 pixel_bits;
> > +	u32 img_fmt;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_in_param - Image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: Indicate the sensor mode. Continuous or single shot.
> > + * @subsample: Indicate to enables SOF subsample or not.
> > + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: Describe input data pattern.
> > + * @raw_pixel_id: Bayer sequence.
> > + * @tg_fps: The fps rate of TG (time generator).
> > + * @img_fmt: The image format of input source.
> > + * @p1_img_crop: The crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	u8 continuous;
> > +	u8 subsample;
> > +	u8 pixel_mode;
> > +	u8 data_pattern;
> > +	u8 raw_pixel_id;
> > +	u16 tg_fps;
> > +	u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - The image output parameters of main stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @pure_raw: Indicate the image path control.
> > + *            True: pure raw
> > + *            False: processing raw
> > + * @pure_raw_pack: Indicate the image is packed or not.
> > + *                 True: packed mode
> > + *                 False: unpacked mode
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	u8 bypass;
> > +	u8 pure_raw;
> > +	u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - The image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_config_param - ISP P1 configuration parameters.
> > + *
> > + * @cfg_in_param: The Image input parameters.
> > + * @cfg_main_param: The main output image parameters.
> > + * @cfg_resize_out_param: The packed output image parameters.
> > + * @enabled_dmas: The enabled DMA port information.
> > + *
> > + */
> > +struct p1_config_param {
> > +	struct cfg_in_param cfg_in_param;
> > +	struct cfg_main_out_param cfg_main_param;
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	u32 enabled_dmas;
> > +} __packed;
> > +
> > +/*
> > + * struct P1_meta_frame - ISP P1 meta frame information.
> > + *
> > + * @enabled_dma: The enabled DMA port information.
> > + * @vb_index: The VB2 index of meta buffer.
> > + * @meta_addr: DMA buffer address of meta buffer.
> > + *
> > + */
> > +struct P1_meta_frame {
> > +	u32 enabled_dma;
> > +	u32 vb_index;
> > +	struct dma_buffer meta_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_init_info - ISP P1 composer init information.
> > + *
> > + * @hw_module: The ISP Camera HW module ID.
> > + * @cq_addr: The DMA address of composer memory.
> > + *
> > + */
> > +struct isp_init_info {
> > +	u8 hw_module;
> > +	struct dma_buffer cq_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_ack_info - ISP P1 IPI command ack information.
> > + *
> > + * @cmd_id: The IPI command ID is acked.
> > + * @frame_seq_no: The IPI frame sequence number is acked.
> > + *
> > + */
> > +struct isp_ack_info {
> > +	u8 cmd_id;
> > +	u32 frame_seq_no;
> > +} __packed;
> > +
> > +/*
> > + * The IPI command enumeration.
> > + */
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +/*
> > + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> > + *
> > + * @cmd_id: The IPI command ID.
> > + * @init_param: The init formation for ISP_CMD_INIT.
> > + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> > + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> > + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> > + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> > + *
> > + */
> > +struct mtk_isp_scp_p1_cmd {
> > +	u8 cmd_id;
> > +	union {
> > +		struct isp_init_info init_param;
> > +		struct p1_config_param config_param;
> > +		u32 enabled_dmas;
> > +		struct P1_meta_frame meta_frame;
> > +		u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +#endif /* __MTK_CAM_IPI_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..ab2277f45fa4
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,95 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_REGS_H__
> > +#define __MTK_CAM_REGS_H__
> > +
> > +/* ISP interrupt enable */
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define DMA_ERR_INT_EN			BIT(29)
> > +
> > +/* ISP interrupt status */
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define VS_INT_ST			BIT(0)
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define HW_PASS1_DON_ST			BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +#define SW_PASS1_DON_ST			BIT(30)
> > +
> > +/* ISP interrupt 2 status */
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +
> > +/* Configures sensor mode */
> > +#define REG_TG_SEN_MODE			0x0230
> > +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_VF_CON			0x0234
> > +#define TG_VF_CON_VFDATA_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_INTER_ST			0x026c
> > +#define TG_CS_MASK			0x3f00
> > +#define TG_IDLE_ST			BIT(8)
> > +
> > +/* IMGO error status register */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +/* RRZO error status register */
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +/* AAO error status register */
> > +#define REG_AAO_ERR_STAT		0x1368
> > +/* AFO error status register */
> > +#define REG_AFO_ERR_STAT		0x136c
> > +/* LCSO error status register */
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +/* BPCI error status register */
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +/* LSCI error status register */
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +/* LMVO error status register */
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +/* FLKO error status register */
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +/* PSO error status register */
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* CQ0 base address */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +/* Frame sequence number */
> > +#define REG_FRAME_SEQ_NUM		0x13b8
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +#endif	/* __MTK_CAM_REGS_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..23fdb8b4abc5
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,2087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_PSO		BIT(10)
> > +
> > +#define MTK_ISP_ONE_PIXEL_MODE		1
> > +#define MTK_ISP_MIN_RESIZE_RATIO	6
> > +#define MTK_ISP_MAX_RUNNING_JOBS	3
> > +
> > +#define MTK_CAM_CIO_PAD_SRC		4
> > +#define MTK_CAM_CIO_PAD_SINK		11
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> > +{
> > +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_request *
> > +mtk_cam_req_to_dev_req(struct media_request *__req)
> > +{
> > +	return container_of(__req, struct mtk_cam_dev_request, req);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> > +				 struct mtk_cam_dev_request *req,
> > +				 enum vb2_buffer_state state)
> > +{
> > +	struct media_request_object *obj, *obj_prev;
> > +	unsigned long flags;
> > +	u64 ts_eof = ktime_get_boottime_ns();
> > +
> > +	if (!cam->streaming)
> > +		return;
> > +
> > +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> > +
> > +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> > +		struct vb2_buffer *vb;
> > +		struct mtk_cam_dev_buffer *buf;
> > +		struct mtk_cam_video_device *node;
> > +
> > +		if (!vb2_request_object_is_buffer(obj))
> > +			continue;
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> > +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +		spin_lock_irqsave(&node->buf_list_lock, flags);
> > +		list_del(&buf->list);
> > +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> > +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > +			vb->timestamp = ts_eof;
> > +		else
> > +			vb->timestamp = req->timestamp;
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +			return req;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +
> > +	return NULL;
> > +}
> > +
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> > +				   unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> > +			list_del(&req->list);
> > +			break;
> > +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> Hi, I see that frame_params.frame_seq_no is incremented when a request is queued
> and frame_seq_no is read from a register, so if the first is lower than the latter
> it means userspace was queueing request too slowly and missed frames right?
> So userspace will have to catch up in order not to get "ERROR" buffers.
> Maybe the driver can just skip lost frames. In the rkisp1 for example,
> if there is no buffer available, the driver just write to a dummy buffer that is
> never sent to userspace. This way userspace always get valid buffers back when
> dequeueing.
> 

Q1. We will update frame_params.frame_seq_no into HW per frame.
So the frame_seq_no is read from HW register should not be large than
frame_params.frame_seq_no. If userspace was queueing request too slowly,
ISP HW just skip frame output due to no available buffers.

Q2. For this code block, it handles the heavy system loading. e.g
abnormal ISR execution. In this scenario, ISP HW will keep output frame
buffers and update frame_seq_no into HW. So ISP P1 driver miss some
interrupt timing and just got the latest frame_seq_no from HW.
With this check, we will return all frame buffers which have be proceed
by HW. Otherwise, userspace may not en-queue any frame buffers due to
some frame buffers are not returned from kernel driver.

> > +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> > +				 req->frame_params.frame_seq_no);
> > +			list_del(&req->list);
> > +		} else {
> > +			break;
> Does this case can ever occur?
> 
> Thanks,
> Dafna
> 

To be honest, it never happened.
I will remove this.

Best regards,

Jungo

> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	if (!cam->streaming) {
> > +		dev_dbg(cam->dev, "stream is off\n");
> > +		return;
> > +	}
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> > +			dev_dbg(cam->dev, "jobs are full\n");
> > +			break;
> > +		}
> > +		cam->running_job_count++;
> > +		list_del(&req->list);
> > +		list_add_tail(&req->list, &cam->running_job_list);
> > +		mtk_isp_req_enqueue(cam, req);
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> > +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> > +					       media_dev);
> > +	unsigned long flags;
> > +
> > +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> > +	vb2_request_queue(req);
> > +
> > +	/* add to pending job list */
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	mtk_cam_dev_req_try_queue(cam);
> > +}
> > +
> > +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			     struct v4l2_pix_format_mplane *mp)
> > +{
> > +	unsigned int bpl, ppl;
> > +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> > +	unsigned int width = mp->width;
> > +
> > +	bpl = 0;
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		/* Bayer encoding format & 2 bytes alignment */
> > +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> > +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		/*
> > +		 * The FULL-G encoding format
> > +		 * 1 G component per pixel
> > +		 * 1 R component per 4 pixel
> > +		 * 1 B component per 4 pixel
> > +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> > +		 */
> > +		ppl = DIV_ROUND_UP(width * 6, 4);
> > +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> > +
> > +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> > +		if (pixel_bits == 10)
> > +			bpl = ALIGN(bpl, 4);
> > +		else
> > +			bpl = ALIGN(bpl, 8);
> > +	}
> > +	/*
> > +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> > +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> > +	 */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	mp->plane_fmt[0].bytesperline = bpl;
> > +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> > +
> > +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> > +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	int i;
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (!queue_desc->image) {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +		return;
> > +	}
> > +
> > +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> > +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> > +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> > +	dest->fmt.pix_mp.num_planes = 1;
> > +
> > +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_R;
> > +	default:
> > +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			  struct p1_img_output *out_fmt, int sd_width,
> > +			  int sd_height)
> > +{
> > +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> > +
> > +	/* Check output & input image size dimension */
> > +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> > +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> > +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> > +			node_id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Check resize ratio for resize out stream due to HW constraint */
> > +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> > +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> > +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> > +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    !out_fmt->pixel_bits) {
> > +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> > +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> > +		return -EINVAL;
> > +	}
> > +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> > +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> > +
> > +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> > +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +	out_fmt->crop.left = 0;
> > +	out_fmt->crop.top = 0;
> > +	out_fmt->crop.width = sd_width;
> > +	out_fmt->crop.height = sd_height;
> > +
> > +	dev_dbg(cam->dev,
> > +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		node_id, out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	cam->enabled_count = 0;
> > +	cam->enabled_dmas = 0;
> > +	cam->stream_count = 0;
> > +	cam->running_job_count = 0;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		if (!cam->vdev_nodes[i].enabled)
> > +			continue;
> > +		cam->enabled_count++;
> > +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> > +	}
> > +
> > +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> > +		cam->enabled_dmas);
> > +}
> > +
> > +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct p1_config_param config_param;
> > +	struct cfg_in_param *cfg_in_param;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	int sd_width, sd_height, sd_code;
> > +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> > +	int ret;
> > +
> > +	/* Get sensor format configuration */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> > +		return ret;
> > +	}
> > +	sd_width = sd_fmt.format.width;
> > +	sd_height = sd_fmt.format.height;
> > +	sd_code = sd_fmt.format.code;
> > +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> > +		sd_code);
> > +
> > +	memset(&config_param, 0, sizeof(config_param));
> > +
> > +	/* Update cfg_in_param */
> > +	cfg_in_param = &config_param.cfg_in_param;
> > +	cfg_in_param->continuous = true;
> > +	/* Fix to one pixel mode in default */
> > +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> > +	cfg_in_param->crop.width = sd_width;
> > +	cfg_in_param->crop.height = sd_height;
> > +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> > +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> > +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> > +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Update cfg_main_param */
> > +	config_param.cfg_main_param.pure_raw = true;
> > +	config_param.cfg_main_param.pure_raw_pack = true;
> > +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> > +			     &config_param.cfg_main_param.output,
> > +			     sd_width, sd_height);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Update cfg_resize_param */
> > +	if (enabled_dma_ports & R_RRZO) {
> > +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> > +				     &config_param.cfg_resize_param.output,
> > +				     sd_width, sd_height);
> > +		if (ret)
> > +			return ret;
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = true;
> > +	}
> > +
> > +	/* Update enabled_dmas */
> > +	config_param.enabled_dmas = enabled_dma_ports;
> > +	mtk_isp_hw_config(cam, &config_param);
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> > +				  unsigned int frame_seq_no)
> > +{
> > +	struct v4l2_event event = {
> > +		.type = V4L2_EVENT_FRAME_SYNC,
> > +		.u.frame_sync.frame_sequence = frame_seq_no,
> > +	};
> > +
> > +	v4l2_event_queue(cam->subdev.devnode, &event);
> > +}
> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> > +{
> > +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> > +	struct device *dev = cam->dev;
> > +	struct media_entity *entity;
> > +	struct v4l2_subdev *sensor;
> > +
> > +	sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> > +			entity->name, entity->function, entity->stream_count);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "no seninf connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "no seninf connected\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> > +	if (!cam->sensor)
> > +		return -ENODEV;
> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		goto fail_seninf_off;
> > +	}
> > +
> > +	ret = mtk_cam_dev_isp_config(cam);
> > +	if (ret)
> > +		goto fail_sensor_off;
> > +
> > +	cam->streaming = true;
> > +	mtk_isp_stream(cam, 1);
> > +	mtk_cam_dev_req_try_queue(cam);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +fail_seninf_off:
> > +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	cam->streaming = false;
> > +	mtk_isp_stream(cam, 0);
> > +	mtk_isp_hw_release(cam);
> > +
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> > +
> > +	if (enable) {
> > +		/* Align vb2_core_streamon design */
> > +		if (cam->streaming) {
> > +			dev_warn(cam->dev, "already streaming on\n");
> > +			return 0;
> > +		}
> > +		return mtk_cam_cio_stream_on(cam);
> > +	}
> > +
> > +	if (!cam->streaming) {
> > +		dev_warn(cam->dev, "already streaming off\n");
> > +		return 0;
> > +	}
> > +	return mtk_cam_cio_stream_off(cam);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	/*
> > +	 * The video nodes exposed by the driver have pads indexes
> > +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> > +	 */
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> > +		cam->vdev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	dma_addr_t addr;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	/* SCP address is only valid for meta input buffer */
> > +	if (!node->desc.smem_alloc)
> > +		return 0;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	/* Use coherent address to get iova address */
> > +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> > +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> > +		return -EFAULT;
> > +	}
> > +	buf->scp_addr = buf->daddr;
> > +	buf->daddr = addr;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size) {
> > +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> > +			vb2_plane_size(vb, 0), size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > +		if (vb2_get_plane_payload(vb, 0) != size) {
> > +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> > +				vb2_get_plane_payload(vb, 0), size);
> > +			return -EINVAL;
> > +		}
> > +		return 0;
> > +	}
> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct device *dev = cam->dev;
> > +
> > +	if (!node->desc.smem_alloc)
> > +		return;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	dma_unmap_page_attrs(dev, buf->daddr,
> > +			     vb->planes[0].length,
> > +			     DMA_BIDIRECTIONAL,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +}
> > +
> > +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (node->desc.smem_alloc)
> > +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size || *num_planes != 1)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> > +		list_del(&buf->list);
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_ret_buf;
> > +	}
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	/* Start streaming of the whole pipeline now*/
> > +	if (!cam->pipeline.streaming_count) {
> > +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> > +		if (ret) {
> > +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> > +			goto fail_unlock;
> > +		}
> > +		mtk_cam_dev_init_stream(cam);
> > +		ret = mtk_isp_hw_init(cam);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init HW:%d\n", ret);
> > +			goto fail_stop_pipeline;
> > +		}
> > +	}
> > +
> > +	/* Media links are fixed after media_pipeline_start */
> > +	cam->stream_count++;
> > +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> > +		cam->enabled_count);
> > +	if (cam->stream_count < cam->enabled_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return 0;
> > +	}
> > +
> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> > +	if (ret)
> > +		goto fail_no_stream;
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	return 0;
> > +
> > +fail_no_stream:
> > +	cam->stream_count--;
> > +fail_stop_pipeline:
> > +	if (cam->stream_count == 0)
> > +		media_pipeline_stop(&node->vdev.entity);
> > +fail_unlock:
> > +	mutex_unlock(&cam->op_lock);
> > +fail_ret_buf:
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> > +		cam->stream_count);
> > +	/* Check the first node to stream-off */
> > +	if (cam->stream_count == cam->enabled_count)
> > +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> > +
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> > +	cam->stream_count--;
> > +	if (cam->stream_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return;
> > +	}
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	mtk_cam_dev_req_cleanup(cam);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> > +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam->dev));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct device *dev = cam->dev;
> > +	const struct v4l2_format *dev_fmt;
> > +	struct v4l2_format try_fmt;
> > +
> > +	memset(&try_fmt, 0, sizeof(try_fmt));
> > +	try_fmt.type = f->type;
> > +
> > +	/* Validate pixelformat */
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> > +	if (!dev_fmt) {
> > +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> > +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> > +	}
> > +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> > +
> > +	/* Validate image width & height range */
> > +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> > +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> > +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> > +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> > +	/* 4 bytes alignment for width */
> > +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> > +
> > +	/* Only support one plane */
> > +	try_fmt.fmt.pix_mp.num_planes = 1;
> > +
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> > +
> > +	/* Constant format fields */
> > +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +
> > +	*f = try_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (vb2_is_busy(node->vdev.queue)) {
> > +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> > +		return -EBUSY;
> > +	}
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +	/* Configure to video device */
> > +	node->vdev_fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> > +	.buf_request_complete = mtk_cam_vb2_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_ops = {
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> > +				  struct media_device *media_dev)
> > +{
> > +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> > +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	media_dev->dev = cam->dev;
> > +	strscpy(media_dev->model, dev_driver_string(dev),
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_ops;
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize subdev pads */
> > +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> > +					sizeof(*cam->subdev_pads),
> > +					GFP_KERNEL);
> > +	if (!cam->subdev_pads) {
> > +		dev_err(dev, "failed to allocate subdev_pads\n");
> > +		ret = -ENOMEM;
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> > +				     cam->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_pads; i++)
> > +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	return 0;
> > +
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> > +			      struct mtk_cam_video_device *node)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> > +	unsigned int link_flags = node->desc.link_flags;
> > +	int ret;
> > +
> > +	/* Initialize mtk_cam_video_device */
> > +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> > +		node->enabled = true;
> > +	else
> > +		node->enabled = false;
> > +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> > +
> > +	cam->subdev_pads[node->id].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->desc.buf_type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam->smem_dev;
> > +	} else {
> > +		vbq->dev = dev;
> > +	}
> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> > +	if (output)
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> > +	else
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> > +	/* No minimum buffers limitation */
> > +	vbq->min_buffers_needed = 0;
> > +	vbq->drv_priv = cam;
> > +	vbq->lock = &node->vdev_lock;
> > +	vbq->supports_requests = true;
> > +	vbq->requires_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_media_clean;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 dev_driver_string(dev), node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->vdev_lock;
> > +	vdev->v4l2_dev = &cam->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> > +	vdev->entity.ops = NULL;
> > +	video_set_drvdata(vdev, cam);
> > +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->vdev_lock);
> > +	INIT_LIST_HEAD(&node->buf_list);
> > +	spin_lock_init(&node->buf_list_lock);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vb2_rel;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam->subdev.entity,
> > +					    node->id, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam->subdev.entity,
> > +					    node->id, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> > +	if (ret)
> > +		goto fail_vdev_ureg;
> > +
> > +	return 0;
> > +
> > +fail_vdev_ureg:
> > +	video_unregister_device(vdev);
> > +fail_vb2_rel:
> > +	mutex_destroy(&node->vdev_lock);
> > +	vb2_queue_release(vbq);
> > +fail_media_clean:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static void
> > +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> > +{
> > +	video_unregister_device(&node->vdev);
> > +	vb2_queue_release(&node->vbq);
> > +	media_entity_cleanup(&node->vdev.entity);
> > +	mutex_destroy(&node->vdev_lock);
> > +}
> > +
> > +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	/* Set up media device & pads */
> > +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> > +	if (ret)
> > +		return ret;
> > +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> > +
> > +	/* Set up v4l2 device */
> > +	cam->v4l2_dev.mdev = &cam->media_dev;
> > +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> > +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> > +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> > +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> > +		 "%s", dev_driver_string(dev));
> > +	v4l2_set_subdevdata(&cam->subdev, cam);
> > +
> > +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> > +		goto fail_clean_media_entiy;
> > +	}
> > +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> > +
> > +		node->id = node->desc.id;
> > +		ret = mtk_cam_video_register_device(cam, node);
> > +		if (ret)
> > +			goto fail_vdev_unreg;
> > +	}
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_vdev_unreg:
> > +	for (i--; i >= 0; i--)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +fail_clean_media_entiy:
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +
> > +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> > +	v4l2_device_unregister_subdev(&cam->subdev);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> > +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	cam->seninf = sd;
> > +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam->seninf = NULL;
> > +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "No seninf subdev\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> > +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> > +				    MEDIA_LNK_FL_IMMUTABLE |
> > +				    MEDIA_LNK_FL_ENABLED);
> > +	if (ret) {
> > +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> > +			cam->seninf->entity.name, cam->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	v4l2_async_notifier_init(&cam->notifier);
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> > +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	v4l2_async_notifier_unregister(&cam->notifier);
> > +	v4l2_async_notifier_cleanup(&cam->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 512 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 1200 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 640 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 288 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	/* This is a default image format */
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> > +{
> > +	unsigned int node_idx;
> > +	int i;
> > +
> > +	node_idx = 0;
> > +	/* Setup the output queue */
> > +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam)
> > +{
> > +	int ret;
> > +
> > +	cam->dev = &pdev->dev;
> > +	mtk_cam_dev_queue_setup(cam);
> > +
> > +	spin_lock_init(&cam->pending_job_lock);
> > +	spin_lock_init(&cam->running_job_lock);
> > +	INIT_LIST_HEAD(&cam->pending_job_list);
> > +	INIT_LIST_HEAD(&cam->running_job_list);
> > +	mutex_init(&cam->op_lock);
> > +
> > +	/* v4l2 sub-device registration */
> > +	ret = mtk_cam_v4l2_register(cam);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam);
> > +	if (ret)
> > +		goto fail_v4l2_unreg;
> > +
> > +	return 0;
> > +
> > +fail_v4l2_unreg:
> > +	mutex_destroy(&cam->op_lock);
> > +	mtk_cam_v4l2_unregister(cam);
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam);
> > +	mtk_cam_v4l2_unregister(cam);
> > +	mutex_destroy(&cam->op_lock);
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..0a340a1e65ea
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,244 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_H__
> > +#define __MTK_CAM_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "mtk_cam-ipi.h"
> > +
> > +#define IMG_MAX_WIDTH		5376
> > +#define IMG_MAX_HEIGHT		4032
> > +#define IMG_MIN_WIDTH		80
> > +#define IMG_MIN_HEIGHT		60
> > +
> > +/*
> > + * ID enum value for struct mtk_cam_dev_node_desc:id
> > + * or mtk_cam_video_device:id
> > + */
> > +enum  {
> > +	MTK_CAM_P1_META_IN_0 = 0,
> > +	MTK_CAM_P1_MAIN_STREAM_OUT,
> > +	MTK_CAM_P1_PACKED_BIN_OUT,
> > +	MTK_CAM_P1_META_OUT_0,
> > +	MTK_CAM_P1_META_OUT_1,
> > +	MTK_CAM_P1_META_OUT_2,
> > +	MTK_CAM_P1_META_OUT_3,
> > +	MTK_CAM_P1_TOTAL_NODES
> > +};
> > +
> > +/* Supported image format list */
> > +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> > +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> > +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> > +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> > +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> > +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> > +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> > +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> > +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> > +
> > +/* Supported bayer pixel order */
> > +#define MTK_CAM_RAW_PXL_ID_B		0
> > +#define MTK_CAM_RAW_PXL_ID_GB		1
> > +#define MTK_CAM_RAW_PXL_ID_GR		2
> > +#define MTK_CAM_RAW_PXL_ID_R		3
> > +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> > +
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +/*
> > + * struct mtk_cam_dev_request - MTK camera device request.
> > + *
> > + * @req: Embedded struct media request.
> > + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> > + * @frame_work: work queue entry for frame transmission to SCP.
> > + * @list: List entry of the object for @struct mtk_cam_dev:
> > + *        pending_job_list or running_job_list.
> > + * @timestamp: Start of frame timestamp in ns
> > + *
> > + */
> > +struct mtk_cam_dev_request {
> > +	struct media_request req;
> > +	struct mtk_p1_frame_param frame_params;
> > +	struct work_struct frame_work;
> > +	struct list_head list;
> > +	u64 timestamp;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> > + *
> > + * @vbb: Embedded struct vb2_v4l2_buffer.
> > + * @list: List entry of the object for @struct mtk_cam_video_device:
> > + *        buf_list.
> > + * @daddr: The DMA address of this buffer.
> > + * @scp_addr: The SCP address of this buffer which
> > + *            is only supported for meta input node.
> > + * @node_id: The vidoe node id which this buffer belongs to.
> > + *
> > + */
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer vbb;
> > +	struct list_head list;
> > +	/* Intenal part */
> > +	dma_addr_t daddr;
> > +	dma_addr_t scp_addr;
> > +	unsigned int node_id;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> > + *
> > + * @id: id of the node
> > + * @name: name of the node
> > + * @cap: supported V4L2 capabilities
> > + * @buf_type: supported V4L2 buffer type
> > + * @dma_port: the dma ports associated to the node
> > + * @link_flags: default media link flags
> > + * @smem_alloc: using the smem_dev as alloc device or not
> > + * @image: true for image node, false for meta node
> > + * @num_fmts: the number of supported node formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum VB2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts: supported format
> > + * @frmsizes: supported V4L2 frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	const char *name;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure
> > + *
> > + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> > + * @enabled: Indicate the video device is enabled or not
> > + * @desc: The node description of video device
> > + * @vdev_fmt: The V4L2 format of video device
> > + * @vdev_pad: The media pad graph object of video device
> > + * @vbq: A videobuf queue of video device
> > + * @vdev: The video device instance
> > + * @vdev_lock: Serializes vb2 queue and video device operations
> > + * @buf_list: List for enqueue buffers
> > + * @buf_list_lock: Lock used to protect buffer list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_format vdev_fmt;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct video_device vdev;
> > +	/* Serializes vb2 queue and video device operations */
> > +	struct mutex vdev_lock;
> > +	struct list_head buf_list;
> > +	/* Lock used to protect buffer list */
> > +	spinlock_t buf_list_lock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @dev: Pointer to device.
> > + * @smem_pdev: Pointer to shared memory device.
> > + * @pipeline: Media pipeline information.
> > + * @media_dev: Media device instance.
> > + * @subdev: The V4L2 sub-device instance.
> > + * @v4l2_dev: The V4L2 device driver instance.
> > + * @notifier: The v4l2_device notifier data.
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> > + * @seninf: Pointer to the seninf sub-device.
> > + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> > + * @streaming: Indicate the overall streaming status is on or off.
> > + * @enabled_dmas: The enabled dma port information when streaming on.
> > + * @enabled_count: Number of enabled video nodes
> > + * @stream_count: Number of streaming video nodes
> > + * @running_job_count: Nunber of running jobs in the HW driver.
> > + * @pending_job_list: List to keep the media requests before en-queue into
> > + *                    HW driver.
> > + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> > + * @running_job_list: List to keep the media requests after en-queue into
> > + *                    HW driver.
> > + * @running_job_lock: Protect the running_job_list data.
> > + * @op_lock: Serializes driver's VB2 callback operations.
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct device *dev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int enabled_dmas;
> > +	unsigned int enabled_count;
> > +	unsigned int stream_count;
> > +	unsigned int running_job_count;
> > +	struct list_head pending_job_list;
> > +	/* Protect the pending_job_list data */
> > +	spinlock_t pending_job_lock;
> > +	struct list_head running_job_list;
> > +	/* Protect the running_job_list data & running_job_count */
> > +	spinlock_t running_job_lock;
> > +	/* Serializes driver's VB2 callback operations */
> > +	struct mutex op_lock;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> > +				   unsigned int frame_seq_no);
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				  unsigned int frame_seq_no);
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no);
> > +
> > +#endif /* __MTK_CAM_H__ */
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-04-09  2:49         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-09  2:49 UTC (permalink / raw)
  To: Dafna Hirschfeld
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport

Hi, Dafna:

Thanks for your comments.

On Thu, 2020-04-02 at 18:45 +0200, Dafna Hirschfeld wrote:
> 
> On 19.12.19 06:49, Jungo Lin wrote:
> > This patch adds the Mediatek ISP P1 HW control device driver.
> > It handles the ISP HW configuration, provides interrupt handling and
> > initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> > implement standard V4L2 video driver that utilizes V4L2 and media
> > framework APIs. It supports one media device, one sub-device and
> > several video devices during initialization. Moreover, it also connects
> > with sensor and seninf drivers with V4L2 async APIs. Communicate with
> > co-process via SCP communication to compose ISP registers in the
> > firmware.
> > 
> > (The current metadata interface used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > ---
> > Changes from v6:
> >   - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >   - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >   - Correct auto suspend timer value for suspend/resume issue
> >   - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >   - Fix KE due to no sen-inf sub-device
> > ---
> >   drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >   .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >   .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >   .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >   .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >   9 files changed, 3377 insertions(+)
> >   create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >   create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> > diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
> > new file mode 100644
> > index 000000000000..f86e1b59ad1e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/Kconfig
> > @@ -0,0 +1,20 @@
> > +config VIDEO_MEDIATEK_ISP_PASS1
> > +	tristate "Mediatek ISP Pass 1 driver"
> > +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> > +	depends on ARCH_MEDIATEK
> > +	select V4L2_FWNODE
> > +	select VIDEOBUF2_VMALLOC
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select MTK_SCP
> > +	default n
> > +	help
> > +		Pass 1 driver controls 3A (auto-focus, exposure,
> > +		and white balance) with tuning feature and outputs
> > +		the captured image buffers in Mediatek's camera system.
> > +
> > +		Choose Y if you want to use Mediatek SoCs to create image
> > +		captured application such as video recording and still image
> > +		capturing.
> > +
> > +		To compile this driver as a module, choose M here; the module
> > +		will be called mtk-cam-isp.
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > new file mode 100644
> > index 000000000000..ce79d283b209
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > new file mode 100644
> > index 000000000000..53b54d3c26a0
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +mtk-cam-isp-objs += mtk_cam.o
> > +mtk-cam-isp-objs += mtk_cam-hw.o
> > +
> > +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
> > \ No newline at end of file
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > new file mode 100644
> > index 000000000000..4065d0d29b7f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> > @@ -0,0 +1,636 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +//
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/atomic.h>
> > +#include <linux/clk.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/module.h>
> > +#include <linux/remoteproc/mtk_scp.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/remoteproc.h>
> > +#include <linux/sched.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/types.h>
> > +#include <linux/videodev2.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include <media/v4l2-event.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +#include "mtk_cam-regs.h"
> > +
> > +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
> > +#define MTK_ISP_CQ_BUFFER_COUNT			3
> > +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
> > +
> > +/*
> > + *
> > + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
> > + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
> > + * For CAM A/C, it only supports max line buffer with 3328 pixels.
> > + * In current driver, only supports CAM B.
> > + *
> > + */
> > +#define MTK_ISP_CAM_ID_B			3
> > +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
> > +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
> > +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
> > +
> > +static void isp_tx_frame_worker(struct work_struct *work)
> > +{
> > +	struct mtk_cam_dev_request *req =
> > +		container_of(work, struct mtk_cam_dev_request, frame_work);
> > +	struct mtk_cam_dev *cam =
> > +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
> > +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_handler(void *data, unsigned int len, void *priv)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_isp_scp_p1_cmd *ipi_msg;
> > +
> > +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
> > +
> > +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
> > +		dev_err(dev, "wrong IPI len:%d\n", len);
> > +		return;
> > +	}
> > +
> > +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
> > +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
> > +		return;
> > +
> > +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
> > +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
> > +}
> > +
> > +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	int ret;
> > +
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI cmd\n");
> > +		return ret;
> > +	}
> > +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
> > +			       isp_composer_handler, p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register IPI frame\n");
> > +		goto unreg_ipi_cmd;
> > +	}
> > +
> > +	p1_dev->composer_wq =
> > +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
> > +					__WQ_LEGACY | WQ_MEM_RECLAIM |
> > +					WQ_FREEZABLE);
> > +	if (!p1_dev->composer_wq) {
> > +		dev_err(dev, "failed to alloc composer workqueue\n");
> > +		goto unreg_ipi_frame;
> > +	}
> > +
> > +	return 0;
> > +
> > +unreg_ipi_frame:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +unreg_ipi_cmd:
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	destroy_workqueue(p1_dev->composer_wq);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
> > +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
> > +}
> > +
> > +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
> > +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
> > +
> > +	/*
> > +	 * Passed coherent reserved memory info. for SCP firmware usage.
> > +	 * This buffer is used for SCP's ISP composer to compose.
> > +	 * The size of is fixed to 0x200000 for the requirement of composer.
> > +	 */
> > +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
> > +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +
> > +	isp_composer_uninit(p1_dev);
> > +}
> > +
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
> > +		       struct p1_config_param *config_param)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
> > +	memcpy(&composer_tx_cmd.config_param, config_param,
> > +	       sizeof(*config_param));
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
> > +{
> > +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
> > +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
> > +	composer_tx_cmd.is_stream_on = on;
> > +
> > +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
> > +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
> > +}
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	ret = rproc_boot(p1_dev->rproc_handle);
> > +	if (ret) {
> > +		dev_err(dev, "failed to rproc_boot\n");
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_composer_init(p1_dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_get_sync(dev);
> > +	isp_composer_hw_init(p1_dev);
> > +
> > +	p1_dev->enqueued_frame_seq_no = 0;
> > +	p1_dev->dequeued_frame_seq_no = 0;
> > +	p1_dev->composed_frame_seq_no = 0;
> > +	p1_dev->sof_count = 0;
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	isp_composer_hw_deinit(p1_dev);
> > +	pm_runtime_mark_last_busy(dev);
> > +	pm_runtime_put_autosuspend(dev);
> > +	rproc_shutdown(p1_dev->rproc_handle);
> > +
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> > +			 struct mtk_cam_dev_request *req)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> > +
> > +	/* Accumulated frame sequence number */
> > +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> > +
> > +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> > +	queue_work(p1_dev->composer_wq, &req->frame_work);
> > +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no,
> > +		cam->running_job_count);
> > +}
> > +
> > +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> > +			       unsigned int dequeued_frame_seq_no)
> > +{
> > +	dma_addr_t base_addr = p1_dev->composer_iova;
> > +	struct device *dev = p1_dev->dev;
> > +	struct mtk_cam_dev_request *req;
> > +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> > +	unsigned int addr_offset;
> > +
> > +	/* Send V4L2_EVENT_FRAME_SYNC event */
> > +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +
> > +	p1_dev->sof_count += 1;
> > +	/* Save frame information */
> > +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> > +
> > +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> > +	if (req)
> > +		req->timestamp = ktime_get_boottime_ns();
> > +
> > +	/* Update CQ base address if needed */
> > +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> > +		dev_dbg(dev,
> > +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> > +			composed_frame_seq_no, dequeued_frame_seq_no);
> > +		return;
> > +	}
> > +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> > +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> > +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> > +	dev_dbg(dev,
> > +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> > +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> > +}
> > +
> > +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	u32 val;
> > +
> > +	dev_err(p1_dev->dev,
> > +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> > +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> > +	dev_err(p1_dev->dev,
> > +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> > +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> > +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> > +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> > +
> > +	/* Disable DMA error mask to avoid too much error log */
> > +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
> > +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
> > +}
> > +
> > +static irqreturn_t isp_irq_cam(int irq, void *data)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
> > +	struct device *dev = p1_dev->dev;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int irq_status, err_status, dma_status;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
> > +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
> > +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
> > +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
> > +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
> > +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
> > +
> > +	/*
> > +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
> > +	 * If these two ISRs come together, print warning msg to hint.
> > +	 */
> > +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
> > +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
> > +
> > +	/* De-queue frame */
> > +	if (irq_status & SW_PASS1_DON_ST) {
> > +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
> > +					      p1_dev->dequeued_frame_seq_no);
> > +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
> > +	}
> > +
> > +	/* Save frame info. & update CQ address for frame HW en-queue */
> > +	if (irq_status & SOF_INT_ST)
> > +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
> > +
> > +	/* Check ISP error status */
> > +	if (err_status) {
> > +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
> > +		/* Show DMA errors in detail */
> > +		if (err_status & DMA_ERR_ST)
> > +			isp_irq_handle_dma_err(p1_dev);
> > +	}
> > +
> > +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
> > +		p1_dev->sof_count, irq_status, dma_status,
> > +		dequeued_frame_seq_no);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
> > +			       struct platform_device *pdev)
> > +{
> > +	struct device *dev = p1_dev->dev;
> > +	dma_addr_t addr;
> > +	void *ptr;
> > +	int ret;
> > +
> > +	p1_dev->scp = scp_get(pdev);
> > +	if (!p1_dev->scp) {
> > +		dev_err(dev, "failed to get scp device\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
> > +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
> > +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
> > +
> > +	/*
> > +	 * Allocate coherent reserved memory for SCP firmware usage.
> > +	 * The size of SCP composer's memory is fixed to 0x200000
> > +	 * for the requirement of firmware.
> > +	 */
> > +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
> > +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
> > +	if (!ptr) {
> > +		ret = -ENOMEM;
> > +		goto fail_put_scp;
> > +	}
> > +
> > +	p1_dev->composer_scp_addr = addr;
> > +	p1_dev->composer_virt_addr = ptr;
> > +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
> > +
> > +	/*
> > +	 * This reserved memory is also be used by ISP P1 HW.
> > +	 * Need to get iova address for ISP P1 DMA.
> > +	 */
> > +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
> > +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map scp iova\n");
> > +		ret = -ENOMEM;
> > +		goto fail_free_mem;
> > +	}
> > +	p1_dev->composer_iova = addr;
> > +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
> > +
> > +	return 0;
> > +
> > +fail_free_mem:
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +fail_put_scp:
> > +	scp_put(p1_dev->scp);
> > +
> > +	return ret;
> > +}
> > +
> > +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
> > +{
> > +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
> > +			  p1_dev->composer_virt_addr,
> > +			  p1_dev->composer_scp_addr);
> > +	p1_dev->composer_scp_addr = 0;
> > +	scp_put(p1_dev->scp);
> > +}
> > +
> > +static int mtk_isp_pm_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Disable ISP's view finder and wait for TG idle if possible */
> > +	dev_dbg(dev, "cam suspend, disable VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
> > +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
> > +				  (val & TG_CS_MASK) == TG_IDLE_ST,
> > +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
> > +
> > +	/* Disable CMOS */
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Force ISP HW to idle */
> > +	ret = pm_runtime_force_suspend(dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to force suspend:%d\n", ret);
> > +		goto reenable_hw;
> > +	}
> > +
> > +	return 0;
> > +
> > +reenable_hw:
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_isp_pm_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	u32 val;
> > +	int ret;
> > +
> > +	dev_dbg(dev, "- %s\n", __func__);
> > +
> > +	if (pm_runtime_suspended(dev))
> > +		return 0;
> > +
> > +	/* Force ISP HW to resume */
> > +	ret = pm_runtime_force_resume(dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Enable CMOS */
> > +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
> > +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
> > +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
> > +
> > +	/* Enable VF */
> > +	val = readl(p1_dev->regs + REG_TG_VF_CON);
> > +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_suspend(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	dev_dbg(dev, "%s:disable clock\n", __func__);
> > +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_runtime_resume(struct device *dev)
> > +{
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +	int ret;
> > +
> > +	dev_dbg(dev, "%s:enable clock\n", __func__);
> > +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to enable clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_probe(struct platform_device *pdev)
> > +{
> > +	/* List of clocks required by isp cam */
> > +	static const char * const clk_names[] = {
> > +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
> > +	};
> > +	struct mtk_isp_p1_device *p1_dev;
> > +	struct device *dev = &pdev->dev;
> > +	struct resource *res;
> > +	int irq, ret, i;
> > +
> > +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
> > +	if (!p1_dev)
> > +		return -ENOMEM;
> > +
> > +	p1_dev->dev = dev;
> > +	dev_set_drvdata(dev, p1_dev);
> > +
> > +	/*
> > +	 * Now only support single CAM with CAM B.
> > +	 * Get CAM B register base with CAM B index.
> > +	 * Support multiple CAMs in future.
> > +	 */
> > +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
> > +	p1_dev->regs = devm_ioremap_resource(dev, res);
> > +	if (IS_ERR(p1_dev->regs)) {
> > +		dev_err(dev, "failed to map reister base\n");
> > +		return PTR_ERR(p1_dev->regs);
> > +	}
> > +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
> > +
> > +	/*
> > +	 * The cam_sys unit only supports reg., but has no IRQ support.
> > +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
> > +	 */
> > +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
> > +	if (!irq) {
> > +		dev_err(dev, "failed to get irq\n");
> > +		return -ENODEV;
> > +	}
> > +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
> > +			       p1_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to request irq=%d\n", irq);
> > +		return ret;
> > +	}
> > +	dev_dbg(dev, "registered irq=%d\n", irq);
> > +	spin_lock_init(&p1_dev->spinlock_irq);
> > +
> > +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
> > +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
> > +				    sizeof(*p1_dev->clks), GFP_KERNEL);
> > +	if (!p1_dev->clks)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < p1_dev->num_clks; ++i)
> > +		p1_dev->clks[i].id = clk_names[i];
> > +
> > +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
> > +	if (ret) {
> > +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = isp_setup_scp_rproc(p1_dev, pdev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
> > +	pm_runtime_use_autosuspend(dev);
> > +	pm_runtime_enable(dev);
> > +
> > +	/* Initialize the v4l2 common part */
> > +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
> > +	if (ret) {
> > +		isp_teardown_scp_rproc(p1_dev);
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_isp_remove(struct platform_device *pdev)
> > +{
> > +	struct device *dev = &pdev->dev;
> > +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> > +
> > +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
> > +	pm_runtime_dont_use_autosuspend(dev);
> > +	pm_runtime_disable(dev);
> > +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
> > +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +	isp_teardown_scp_rproc(p1_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_isp_pm_ops = {
> > +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
> > +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
> > +			   NULL)
> > +};
> > +
> > +static const struct of_device_id mtk_isp_of_ids[] = {
> > +	{.compatible = "mediatek,mt8183-camisp",},
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
> > +
> > +static struct platform_driver mtk_isp_driver = {
> > +	.probe   = mtk_isp_probe,
> > +	.remove  = mtk_isp_remove,
> > +	.driver  = {
> > +		.name  = "mtk-cam-p1",
> > +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
> > +		.pm     = &mtk_isp_pm_ops,
> > +	}
> > +};
> > +
> > +module_platform_driver(mtk_isp_driver);
> > +
> > +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
> > +MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > new file mode 100644
> > index 000000000000..837662f92a5e
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> > @@ -0,0 +1,64 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_HW_H__
> > +#define __MTK_CAM_HW_H__
> > +
> > +#include <linux/types.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-ipi.h"
> > +
> > +/*
> > + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
> > + *
> > + * @dev: Pointer to device.
> > + * @scp_pdev: Pointer to SCP platform device.
> > + * @rproc_handle: Pointer to new remoteproc instance.
> > + * @cam_dev: Embedded struct cam_dev
> > + * @regs: Camera ISP HW base register address
> > + * @num_clks: The number of driver's clocks
> > + * @clks: The clock data array
> > + * @spinlock_irq: Used to protect register read/write data
> > + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
> > + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
> > + * @composed_frame_seq_no: Frame sequence number of composed frame
> > + * @timestamp: Frame timestamp in ns
> > + * @sof_count: SOF counter
> > + * @composer_wq: The work queue for frame request composing
> > + * @composer_scp_addr: SCP address of ISP composer memory
> > + * @composer_iova: DMA address of ISP composer memory
> > + * @virt_addr: Virtual address of ISP composer memory
> > + *
> > + */
> > +struct mtk_isp_p1_device {
> > +	struct device *dev;
> > +	struct mtk_scp *scp;
> > +	struct rproc *rproc_handle;
> > +	struct mtk_cam_dev cam_dev;
> > +	void __iomem *regs;
> > +	unsigned int num_clks;
> > +	struct clk_bulk_data *clks;
> > +	/* Used to protect register read/write data */
> > +	spinlock_t spinlock_irq;
> > +	unsigned int enqueued_frame_seq_no;
> > +	unsigned int dequeued_frame_seq_no;
> > +	unsigned int composed_frame_seq_no;
> > +	u8 sof_count;
> > +	struct workqueue_struct *composer_wq;
> > +	dma_addr_t composer_scp_addr;
> > +	dma_addr_t composer_iova;
> > +	void *composer_virt_addr;
> > +};
> > +
> > +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
> > +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
> > +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
> > +		       struct p1_config_param *config_param);
> > +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
> > +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
> > +			 struct mtk_cam_dev_request *req);
> > +
> > +#endif /* __MTK_CAM_HW_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > new file mode 100644
> > index 000000000000..981b634dd91f
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> > @@ -0,0 +1,222 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_IPI_H__
> > +#define __MTK_CAM_IPI_H__
> > +
> > +#include <linux/types.h>
> > +
> > +/*
> > + * struct img_size - Image size information.
> > + *
> > + * @w: Image width, the unit is pixel
> > + * @h: Image height, the unit is pixel
> > + * @xsize: Bytes per line based on width.
> > + * @stride: Bytes per line when changing line.
> > + *          Stride is based on xsize + HW constrain(byte align).
> > + *
> > + */
> > +struct img_size {
> > +	u32 w;
> > +	u32 h;
> > +	u32 xsize;
> > +	u32 stride;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_crop - image corp information
> > + *
> > + * @left: The left of crop area.
> > + * @top: The top of crop area.
> > + * @width: The width of crop area.
> > + * @height: The height of crop area.
> > + *
> > + */
> > +struct p1_img_crop {
> > +	u32 left;
> > +	u32 top;
> > +	u32 width;
> > +	u32 height;
> > +} __packed;
> > +
> > +/*
> > + * struct dma_buffer - DMA buffer address information
> > + *
> > + * @iova: DMA address for ISP DMA device
> > + * @scp_addr: SCP address for external co-process unit
> > + *
> > + */
> > +struct dma_buffer {
> > +	u32 iova;
> > +	u32 scp_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_img_output - ISP P1 image output information
> > + *
> > + * @buffer: DMA buffer address of image.
> > + * @size: The image size configuration.
> > + * @crop: The crop configuration.
> > + * @pixel_bits: The bits per image pixel.
> > + * @img_fmt: The image format.
> > + *
> > + */
> > +struct p1_img_output {
> > +	struct dma_buffer buffer;
> > +	struct img_size size;
> > +	struct p1_img_crop crop;
> > +	u8 pixel_bits;
> > +	u32 img_fmt;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_in_param - Image input parameters structure.
> > + *                       Normally, it comes from sensor information.
> > + *
> > + * @continuous: Indicate the sensor mode. Continuous or single shot.
> > + * @subsample: Indicate to enables SOF subsample or not.
> > + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
> > + * @data_pattern: Describe input data pattern.
> > + * @raw_pixel_id: Bayer sequence.
> > + * @tg_fps: The fps rate of TG (time generator).
> > + * @img_fmt: The image format of input source.
> > + * @p1_img_crop: The crop configuration of input source.
> > + *
> > + */
> > +struct cfg_in_param {
> > +	u8 continuous;
> > +	u8 subsample;
> > +	u8 pixel_mode;
> > +	u8 data_pattern;
> > +	u8 raw_pixel_id;
> > +	u16 tg_fps;
> > +	u32 img_fmt;
> > +	struct p1_img_crop crop;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_main_out_param - The image output parameters of main stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @pure_raw: Indicate the image path control.
> > + *            True: pure raw
> > + *            False: processing raw
> > + * @pure_raw_pack: Indicate the image is packed or not.
> > + *                 True: packed mode
> > + *                 False: unpacked mode
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_main_out_param {
> > +	u8 bypass;
> > +	u8 pure_raw;
> > +	u8 pure_raw_pack;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct cfg_resize_out_param - The image output parameters of
> > + *                               packed out stream.
> > + *
> > + * @bypass: Indicate this device is enabled or disabled or not.
> > + * @p1_img_output: The output image information.
> > + *
> > + */
> > +struct cfg_resize_out_param {
> > +	u8 bypass;
> > +	struct p1_img_output output;
> > +} __packed;
> > +
> > +/*
> > + * struct p1_config_param - ISP P1 configuration parameters.
> > + *
> > + * @cfg_in_param: The Image input parameters.
> > + * @cfg_main_param: The main output image parameters.
> > + * @cfg_resize_out_param: The packed output image parameters.
> > + * @enabled_dmas: The enabled DMA port information.
> > + *
> > + */
> > +struct p1_config_param {
> > +	struct cfg_in_param cfg_in_param;
> > +	struct cfg_main_out_param cfg_main_param;
> > +	struct cfg_resize_out_param cfg_resize_param;
> > +	u32 enabled_dmas;
> > +} __packed;
> > +
> > +/*
> > + * struct P1_meta_frame - ISP P1 meta frame information.
> > + *
> > + * @enabled_dma: The enabled DMA port information.
> > + * @vb_index: The VB2 index of meta buffer.
> > + * @meta_addr: DMA buffer address of meta buffer.
> > + *
> > + */
> > +struct P1_meta_frame {
> > +	u32 enabled_dma;
> > +	u32 vb_index;
> > +	struct dma_buffer meta_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_init_info - ISP P1 composer init information.
> > + *
> > + * @hw_module: The ISP Camera HW module ID.
> > + * @cq_addr: The DMA address of composer memory.
> > + *
> > + */
> > +struct isp_init_info {
> > +	u8 hw_module;
> > +	struct dma_buffer cq_addr;
> > +} __packed;
> > +
> > +/*
> > + * struct isp_ack_info - ISP P1 IPI command ack information.
> > + *
> > + * @cmd_id: The IPI command ID is acked.
> > + * @frame_seq_no: The IPI frame sequence number is acked.
> > + *
> > + */
> > +struct isp_ack_info {
> > +	u8 cmd_id;
> > +	u32 frame_seq_no;
> > +} __packed;
> > +
> > +/*
> > + * The IPI command enumeration.
> > + */
> > +enum mtk_isp_scp_cmds {
> > +	ISP_CMD_INIT,
> > +	ISP_CMD_CONFIG,
> > +	ISP_CMD_STREAM,
> > +	ISP_CMD_DEINIT,
> > +	ISP_CMD_ACK,
> > +	ISP_CMD_FRAME_ACK,
> > +	ISP_CMD_RESERVED,
> > +};
> > +
> > +/*
> > + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
> > + *
> > + * @cmd_id: The IPI command ID.
> > + * @init_param: The init formation for ISP_CMD_INIT.
> > + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
> > + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
> > + * @is_stream_on: The stream information for ISP_CMD_STREAM.
> > + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
> > + *
> > + */
> > +struct mtk_isp_scp_p1_cmd {
> > +	u8 cmd_id;
> > +	union {
> > +		struct isp_init_info init_param;
> > +		struct p1_config_param config_param;
> > +		u32 enabled_dmas;
> > +		struct P1_meta_frame meta_frame;
> > +		u8 is_stream_on;
> > +		struct isp_ack_info ack_info;
> > +	};
> > +} __packed;
> > +
> > +#endif /* __MTK_CAM_IPI_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > new file mode 100644
> > index 000000000000..ab2277f45fa4
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> > @@ -0,0 +1,95 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_REGS_H__
> > +#define __MTK_CAM_REGS_H__
> > +
> > +/* ISP interrupt enable */
> > +#define REG_CTL_RAW_INT_EN		0x0020
> > +#define DMA_ERR_INT_EN			BIT(29)
> > +
> > +/* ISP interrupt status */
> > +#define REG_CTL_RAW_INT_STAT		0x0024
> > +#define VS_INT_ST			BIT(0)
> > +#define TG_ERR_ST			BIT(4)
> > +#define TG_GBERR_ST			BIT(5)
> > +#define CQ_CODE_ERR_ST			BIT(6)
> > +#define CQ_APB_ERR_ST			BIT(7)
> > +#define CQ_VS_ERR_ST			BIT(8)
> > +#define HW_PASS1_DON_ST			BIT(11)
> > +#define SOF_INT_ST			BIT(12)
> > +#define AMX_ERR_ST			BIT(15)
> > +#define RMX_ERR_ST			BIT(16)
> > +#define BMX_ERR_ST			BIT(17)
> > +#define RRZO_ERR_ST			BIT(18)
> > +#define AFO_ERR_ST			BIT(19)
> > +#define IMGO_ERR_ST			BIT(20)
> > +#define AAO_ERR_ST			BIT(21)
> > +#define PSO_ERR_ST			BIT(22)
> > +#define LCSO_ERR_ST			BIT(23)
> > +#define BNR_ERR_ST			BIT(24)
> > +#define LSCI_ERR_ST			BIT(25)
> > +#define DMA_ERR_ST			BIT(29)
> > +#define SW_PASS1_DON_ST			BIT(30)
> > +
> > +/* ISP interrupt 2 status */
> > +#define REG_CTL_RAW_INT2_STAT		0x0034
> > +#define AFO_DONE_ST			BIT(5)
> > +#define AAO_DONE_ST			BIT(7)
> > +
> > +/* Configures sensor mode */
> > +#define REG_TG_SEN_MODE			0x0230
> > +#define TG_SEN_MODE_CMOS_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_VF_CON			0x0234
> > +#define TG_VF_CON_VFDATA_EN		BIT(0)
> > +
> > +/* View finder mode control */
> > +#define REG_TG_INTER_ST			0x026c
> > +#define TG_CS_MASK			0x3f00
> > +#define TG_IDLE_ST			BIT(8)
> > +
> > +/* IMGO error status register */
> > +#define REG_IMGO_ERR_STAT		0x1360
> > +/* RRZO error status register */
> > +#define REG_RRZO_ERR_STAT		0x1364
> > +/* AAO error status register */
> > +#define REG_AAO_ERR_STAT		0x1368
> > +/* AFO error status register */
> > +#define REG_AFO_ERR_STAT		0x136c
> > +/* LCSO error status register */
> > +#define REG_LCSO_ERR_STAT		0x1370
> > +/* BPCI error status register */
> > +#define REG_BPCI_ERR_STAT		0x137c
> > +/* LSCI error status register */
> > +#define REG_LSCI_ERR_STAT		0x1384
> > +/* LMVO error status register */
> > +#define REG_LMVO_ERR_STAT		0x1390
> > +/* FLKO error status register */
> > +#define REG_FLKO_ERR_STAT		0x1394
> > +/* PSO error status register */
> > +#define REG_PSO_ERR_STAT		0x13a0
> > +
> > +/* CQ0 base address */
> > +#define REG_CQ_THR0_BASEADDR		0x0198
> > +/* Frame sequence number */
> > +#define REG_FRAME_SEQ_NUM		0x13b8
> > +
> > +/* IRQ Error Mask */
> > +#define INT_ST_MASK_CAM_ERR		( \
> > +					TG_ERR_ST |\
> > +					TG_GBERR_ST |\
> > +					CQ_CODE_ERR_ST |\
> > +					CQ_APB_ERR_ST |\
> > +					CQ_VS_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					RMX_ERR_ST |\
> > +					BMX_ERR_ST |\
> > +					BNR_ERR_ST |\
> > +					LSCI_ERR_ST |\
> > +					DMA_ERR_ST)
> > +
> > +#endif	/* __MTK_CAM_REGS_H__ */
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > new file mode 100644
> > index 000000000000..23fdb8b4abc5
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> > @@ -0,0 +1,2087 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (c) 2019 MediaTek Inc.
> > +
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/videodev2.h>
> > +#include <media/media-entity.h>
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "mtk_cam.h"
> > +#include "mtk_cam-hw.h"
> > +
> > +#define R_IMGO		BIT(0)
> > +#define R_RRZO		BIT(1)
> > +#define R_AAO		BIT(3)
> > +#define R_AFO		BIT(4)
> > +#define R_LCSO		BIT(5)
> > +#define R_LMVO		BIT(7)
> > +#define R_FLKO		BIT(8)
> > +#define R_PSO		BIT(10)
> > +
> > +#define MTK_ISP_ONE_PIXEL_MODE		1
> > +#define MTK_ISP_MIN_RESIZE_RATIO	6
> > +#define MTK_ISP_MAX_RUNNING_JOBS	3
> > +
> > +#define MTK_CAM_CIO_PAD_SRC		4
> > +#define MTK_CAM_CIO_PAD_SINK		11
> > +
> > +static inline struct mtk_cam_video_device *
> > +file_to_mtk_cam_node(struct file *__file)
> > +{
> > +	return container_of(video_devdata(__file),
> > +		struct mtk_cam_video_device, vdev);
> > +}
> > +
> > +static inline struct mtk_cam_video_device *
> > +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
> > +{
> > +	return container_of(__vq, struct mtk_cam_video_device, vbq);
> > +}
> > +
> > +static inline struct mtk_cam_dev_request *
> > +mtk_cam_req_to_dev_req(struct media_request *__req)
> > +{
> > +	return container_of(__req, struct mtk_cam_dev_request, req);
> > +}
> > +
> > +static inline struct mtk_cam_dev_buffer *
> > +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
> > +{
> > +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
> > +}
> > +
> > +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
> > +				 struct mtk_cam_dev_request *req,
> > +				 enum vb2_buffer_state state)
> > +{
> > +	struct media_request_object *obj, *obj_prev;
> > +	unsigned long flags;
> > +	u64 ts_eof = ktime_get_boottime_ns();
> > +
> > +	if (!cam->streaming)
> > +		return;
> > +
> > +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> > +		req->req.debug_str, req->frame_params.frame_seq_no, state);
> > +
> > +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> > +		struct vb2_buffer *vb;
> > +		struct mtk_cam_dev_buffer *buf;
> > +		struct mtk_cam_video_device *node;
> > +
> > +		if (!vb2_request_object_is_buffer(obj))
> > +			continue;
> > +		vb = container_of(obj, struct vb2_buffer, req_obj);
> > +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +		spin_lock_irqsave(&node->buf_list_lock, flags);
> > +		list_del(&buf->list);
> > +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +		buf->vbb.sequence = req->frame_params.frame_seq_no;
> > +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > +			vb->timestamp = ts_eof;
> > +		else
> > +			vb->timestamp = req->timestamp;
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +}
> > +
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +			return req;
> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +
> > +	return NULL;
> > +}
> > +
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> > +				   unsigned int frame_seq_no)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> > +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> > +			req->frame_params.frame_seq_no, frame_seq_no);
> > +
> > +		/* Match by the en-queued request number */
> > +		if (req->frame_params.frame_seq_no == frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> > +			list_del(&req->list);
> > +			break;
> > +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> > +			cam->running_job_count--;
> > +			/* Pass to user space for frame drop */
> > +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> Hi, I see that frame_params.frame_seq_no is incremented when a request is queued
> and frame_seq_no is read from a register, so if the first is lower than the latter
> it means userspace was queueing request too slowly and missed frames right?
> So userspace will have to catch up in order not to get "ERROR" buffers.
> Maybe the driver can just skip lost frames. In the rkisp1 for example,
> if there is no buffer available, the driver just write to a dummy buffer that is
> never sent to userspace. This way userspace always get valid buffers back when
> dequeueing.
> 

Q1. We will update frame_params.frame_seq_no into HW per frame.
So the frame_seq_no is read from HW register should not be large than
frame_params.frame_seq_no. If userspace was queueing request too slowly,
ISP HW just skip frame output due to no available buffers.

Q2. For this code block, it handles the heavy system loading. e.g
abnormal ISR execution. In this scenario, ISP HW will keep output frame
buffers and update frame_seq_no into HW. So ISP P1 driver miss some
interrupt timing and just got the latest frame_seq_no from HW.
With this check, we will return all frame buffers which have be proceed
by HW. Otherwise, userspace may not en-queue any frame buffers due to
some frame buffers are not returned from kernel driver.

> > +			dev_warn(cam->dev, "frame_seq:%d drop\n",
> > +				 req->frame_params.frame_seq_no);
> > +			list_del(&req->list);
> > +		} else {
> > +			break;
> Does this case can ever occur?
> 
> Thanks,
> Dafna
> 

To be honest, it never happened.
I will remove this.

Best regards,

Jungo

> > +		}
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
> > +		list_del(&req->list);
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +}
> > +
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
> > +{
> > +	struct mtk_cam_dev_request *req, *req_prev;
> > +	unsigned long flags;
> > +
> > +	if (!cam->streaming) {
> > +		dev_dbg(cam->dev, "stream is off\n");
> > +		return;
> > +	}
> > +
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	spin_lock_irqsave(&cam->running_job_lock, flags);
> > +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
> > +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
> > +			dev_dbg(cam->dev, "jobs are full\n");
> > +			break;
> > +		}
> > +		cam->running_job_count++;
> > +		list_del(&req->list);
> > +		list_add_tail(&req->list, &cam->running_job_list);
> > +		mtk_isp_req_enqueue(cam, req);
> > +	}
> > +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +}
> > +
> > +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req;
> > +
> > +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
> > +
> > +	return &cam_dev_req->req;
> > +}
> > +
> > +static void mtk_cam_req_free(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
> > +
> > +	kfree(cam_dev_req);
> > +}
> > +
> > +static void mtk_cam_req_queue(struct media_request *req)
> > +{
> > +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
> > +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
> > +					       media_dev);
> > +	unsigned long flags;
> > +
> > +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
> > +	vb2_request_queue(req);
> > +
> > +	/* add to pending job list */
> > +	spin_lock_irqsave(&cam->pending_job_lock, flags);
> > +	list_add_tail(&cam_req->list, &cam->pending_job_list);
> > +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
> > +
> > +	mtk_cam_dev_req_try_queue(cam);
> > +}
> > +
> > +static unsigned int get_pixel_bits(unsigned int pix_fmt)
> > +{
> > +	switch (pix_fmt) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return 8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return 10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return 12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return 14;
> > +	default:
> > +		return 0;
> > +	}
> > +}
> > +
> > +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			     struct v4l2_pix_format_mplane *mp)
> > +{
> > +	unsigned int bpl, ppl;
> > +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> > +	unsigned int width = mp->width;
> > +
> > +	bpl = 0;
> > +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
> > +		/* Bayer encoding format & 2 bytes alignment */
> > +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
> > +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
> > +		/*
> > +		 * The FULL-G encoding format
> > +		 * 1 G component per pixel
> > +		 * 1 R component per 4 pixel
> > +		 * 1 B component per 4 pixel
> > +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
> > +		 */
> > +		ppl = DIV_ROUND_UP(width * 6, 4);
> > +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
> > +
> > +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
> > +		if (pixel_bits == 10)
> > +			bpl = ALIGN(bpl, 4);
> > +		else
> > +			bpl = ALIGN(bpl, 8);
> > +	}
> > +	/*
> > +	 * This image output buffer will be input buffer of MTK CAM DIP HW
> > +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
> > +	 */
> > +	bpl = ALIGN(bpl, 4);
> > +
> > +	mp->plane_fmt[0].bytesperline = bpl;
> > +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
> > +
> > +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
> > +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
> > +}
> > +
> > +static const struct v4l2_format *
> > +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
> > +{
> > +	int i;
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	for (i = 0; i < desc->num_fmts; i++) {
> > +		dev_fmt = &desc->fmts[i];
> > +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
> > +			return dev_fmt;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +/* Get the default format setting */
> > +static void
> > +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
> > +			     struct mtk_cam_dev_node_desc *queue_desc,
> > +			     struct v4l2_format *dest)
> > +{
> > +	const struct v4l2_format *default_fmt =
> > +		&queue_desc->fmts[queue_desc->default_fmt_idx];
> > +
> > +	dest->type = queue_desc->buf_type;
> > +
> > +	/* Configure default format based on node type */
> > +	if (!queue_desc->image) {
> > +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
> > +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
> > +		return;
> > +	}
> > +
> > +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
> > +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
> > +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
> > +	dest->fmt.pix_mp.num_planes = 1;
> > +
> > +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +}
> > +
> > +/* Utility functions */
> > +static unsigned int get_sensor_pixel_id(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_B;
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GB;
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_GR;
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_RAW_PXL_ID_R;
> > +	default:
> > +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_sensor_fmt(unsigned int fmt)
> > +{
> > +	switch (fmt) {
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case MEDIA_BUS_FMT_SBGGR12_1X12:
> > +	case MEDIA_BUS_FMT_SGBRG12_1X12:
> > +	case MEDIA_BUS_FMT_SGRBG12_1X12:
> > +	case MEDIA_BUS_FMT_SRGGB12_1X12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case MEDIA_BUS_FMT_SBGGR14_1X14:
> > +	case MEDIA_BUS_FMT_SGBRG14_1X14:
> > +	case MEDIA_BUS_FMT_SGRBG14_1X14:
> > +	case MEDIA_BUS_FMT_SRGGB14_1X14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static unsigned int get_img_fmt(unsigned int fourcc)
> > +{
> > +	switch (fourcc) {
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8:
> > +		return MTK_CAM_IMG_FMT_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER8;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10:
> > +		return MTK_CAM_IMG_FMT_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER10;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12:
> > +		return MTK_CAM_IMG_FMT_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER12;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14:
> > +		return MTK_CAM_IMG_FMT_BAYER14;
> > +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
> > +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
> > +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
> > +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
> > +		return MTK_CAM_IMG_FMT_FG_BAYER14;
> > +	default:
> > +		return MTK_CAM_IMG_FMT_UNKNOWN;
> > +	}
> > +}
> > +
> > +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
> > +			  struct p1_img_output *out_fmt, int sd_width,
> > +			  int sd_height)
> > +{
> > +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
> > +
> > +	/* Check output & input image size dimension */
> > +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
> > +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
> > +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
> > +			node_id);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Check resize ratio for resize out stream due to HW constraint */
> > +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO) ||
> > +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
> > +	    MTK_ISP_MIN_RESIZE_RATIO)) {
> > +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
> > +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
> > +		return -EINVAL;
> > +	}
> > +
> > +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
> > +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    !out_fmt->pixel_bits) {
> > +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
> > +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
> > +		return -EINVAL;
> > +	}
> > +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
> > +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
> > +
> > +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
> > +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
> > +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
> > +
> > +	out_fmt->crop.left = 0;
> > +	out_fmt->crop.top = 0;
> > +	out_fmt->crop.width = sd_width;
> > +	out_fmt->crop.height = sd_height;
> > +
> > +	dev_dbg(cam->dev,
> > +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
> > +		node_id, out_fmt->size.w, out_fmt->size.h,
> > +		out_fmt->size.stride, out_fmt->size.xsize,
> > +		out_fmt->crop.width, out_fmt->crop.height);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	cam->enabled_count = 0;
> > +	cam->enabled_dmas = 0;
> > +	cam->stream_count = 0;
> > +	cam->running_job_count = 0;
> > +
> > +	/* Get the enabled meta DMA ports */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		if (!cam->vdev_nodes[i].enabled)
> > +			continue;
> > +		cam->enabled_count++;
> > +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
> > +	}
> > +
> > +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
> > +		cam->enabled_dmas);
> > +}
> > +
> > +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct p1_config_param config_param;
> > +	struct cfg_in_param *cfg_in_param;
> > +	struct v4l2_subdev_format sd_fmt;
> > +	int sd_width, sd_height, sd_code;
> > +	unsigned int enabled_dma_ports = cam->enabled_dmas;
> > +	int ret;
> > +
> > +	/* Get sensor format configuration */
> > +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> > +	if (ret) {
> > +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> > +		return ret;
> > +	}
> > +	sd_width = sd_fmt.format.width;
> > +	sd_height = sd_fmt.format.height;
> > +	sd_code = sd_fmt.format.code;
> > +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> > +		sd_code);
> > +
> > +	memset(&config_param, 0, sizeof(config_param));
> > +
> > +	/* Update cfg_in_param */
> > +	cfg_in_param = &config_param.cfg_in_param;
> > +	cfg_in_param->continuous = true;
> > +	/* Fix to one pixel mode in default */
> > +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
> > +	cfg_in_param->crop.width = sd_width;
> > +	cfg_in_param->crop.height = sd_height;
> > +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
> > +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
> > +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
> > +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
> > +		dev_err(dev, "unknown sd code:%d\n", sd_code);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Update cfg_main_param */
> > +	config_param.cfg_main_param.pure_raw = true;
> > +	config_param.cfg_main_param.pure_raw_pack = true;
> > +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
> > +			     &config_param.cfg_main_param.output,
> > +			     sd_width, sd_height);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* Update cfg_resize_param */
> > +	if (enabled_dma_ports & R_RRZO) {
> > +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
> > +				     &config_param.cfg_resize_param.output,
> > +				     sd_width, sd_height);
> > +		if (ret)
> > +			return ret;
> > +	} else {
> > +		config_param.cfg_resize_param.bypass = true;
> > +	}
> > +
> > +	/* Update enabled_dmas */
> > +	config_param.enabled_dmas = enabled_dma_ports;
> > +	mtk_isp_hw_config(cam, &config_param);
> > +	dev_dbg(dev, "%s done\n", __func__);
> > +
> > +	return 0;
> > +}
> > +
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
> > +				  unsigned int frame_seq_no)
> > +{
> > +	struct v4l2_event event = {
> > +		.type = V4L2_EVENT_FRAME_SYNC,
> > +		.u.frame_sync.frame_sequence = frame_seq_no,
> > +	};
> > +
> > +	v4l2_event_queue(cam->subdev.devnode, &event);
> > +}
> > +
> > +static struct v4l2_subdev *
> > +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
> > +{
> > +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
> > +	struct device *dev = cam->dev;
> > +	struct media_entity *entity;
> > +	struct v4l2_subdev *sensor;
> > +
> > +	sensor = NULL;
> > +	media_device_for_each_entity(entity, mdev) {
> > +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
> > +			entity->name, entity->function, entity->stream_count);
> > +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
> > +		    entity->stream_count) {
> > +			sensor = media_entity_to_v4l2_subdev(entity);
> > +			dev_dbg(dev, "sensor found: %s\n", entity->name);
> > +			break;
> > +		}
> > +	}
> > +
> > +	if (!sensor)
> > +		dev_err(dev, "no seninf connected\n");
> > +
> > +	return sensor;
> > +}
> > +
> > +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "no seninf connected\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	/* Get active sensor from graph topology */
> > +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
> > +	if (!cam->sensor)
> > +		return -ENODEV;
> > +
> > +	/* Seninf must stream on first */
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream on %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		goto fail_seninf_off;
> > +	}
> > +
> > +	ret = mtk_cam_dev_isp_config(cam);
> > +	if (ret)
> > +		goto fail_sensor_off;
> > +
> > +	cam->streaming = true;
> > +	mtk_isp_stream(cam, 1);
> > +	mtk_cam_dev_req_try_queue(cam);
> > +	dev_dbg(dev, "streamed on Pass 1\n");
> > +
> > +	return 0;
> > +
> > +fail_sensor_off:
> > +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +fail_seninf_off:
> > +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->sensor->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
> > +	if (ret) {
> > +		dev_err(dev, "failed to stream off %s:%d\n",
> > +			cam->seninf->entity.name, ret);
> > +		return -EPERM;
> > +	}
> > +
> > +	cam->streaming = false;
> > +	mtk_isp_stream(cam, 0);
> > +	mtk_isp_hw_release(cam);
> > +
> > +	dev_dbg(dev, "streamed off Pass 1\n");
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > +{
> > +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
> > +
> > +	if (enable) {
> > +		/* Align vb2_core_streamon design */
> > +		if (cam->streaming) {
> > +			dev_warn(cam->dev, "already streaming on\n");
> > +			return 0;
> > +		}
> > +		return mtk_cam_cio_stream_on(cam);
> > +	}
> > +
> > +	if (!cam->streaming) {
> > +		dev_warn(cam->dev, "already streaming off\n");
> > +		return 0;
> > +	}
> > +	return mtk_cam_cio_stream_off(cam);
> > +}
> > +
> > +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
> > +				      struct v4l2_fh *fh,
> > +				      struct v4l2_event_subscription *sub)
> > +{
> > +	switch (sub->type) {
> > +	case V4L2_EVENT_FRAME_SYNC:
> > +		return v4l2_event_subscribe(fh, sub, 0, NULL);
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +}
> > +
> > +static int mtk_cam_media_link_setup(struct media_entity *entity,
> > +				    const struct media_pad *local,
> > +				    const struct media_pad *remote, u32 flags)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(entity, struct mtk_cam_dev, subdev.entity);
> > +	u32 pad = local->index;
> > +
> > +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
> > +		__func__, pad, remote->index, flags);
> > +
> > +	/*
> > +	 * The video nodes exposed by the driver have pads indexes
> > +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
> > +	 */
> > +	if (pad < MTK_CAM_P1_TOTAL_NODES)
> > +		cam->vdev_nodes[pad].enabled =
> > +			!!(flags & MEDIA_LNK_FL_ENABLED);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> > +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> > +
> > +	/* added the buffer into the tracking list */
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_add_tail(&buf->list, &node->buf_list);
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +
> > +	/* update buffer internal address */
> > +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> > +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct device *dev = cam->dev;
> > +	struct mtk_cam_dev_buffer *buf;
> > +	dma_addr_t addr;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	buf->node_id = node->id;
> > +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> > +	buf->scp_addr = 0;
> > +
> > +	/* SCP address is only valid for meta input buffer */
> > +	if (!node->desc.smem_alloc)
> > +		return 0;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	/* Use coherent address to get iova address */
> > +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> > +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
> > +	if (dma_mapping_error(dev, addr)) {
> > +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> > +		return -EFAULT;
> > +	}
> > +	buf->scp_addr = buf->daddr;
> > +	buf->daddr = addr;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	if (vb2_plane_size(vb, 0) < size) {
> > +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> > +			vb2_plane_size(vb, 0), size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > +		if (vb2_get_plane_payload(vb, 0) != size) {
> > +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> > +				vb2_get_plane_payload(vb, 0), size);
> > +			return -EINVAL;
> > +		}
> > +		return 0;
> > +	}
> > +
> > +	v4l2_buf->field = V4L2_FIELD_NONE;
> > +	vb2_set_plane_payload(vb, 0, size);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct mtk_cam_dev_buffer *buf;
> > +	struct device *dev = cam->dev;
> > +
> > +	if (!node->desc.smem_alloc)
> > +		return;
> > +
> > +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> > +	dma_unmap_page_attrs(dev, buf->daddr,
> > +			     vb->planes[0].length,
> > +			     DMA_BIDIRECTIONAL,
> > +			     DMA_ATTR_SKIP_CPU_SYNC);
> > +}
> > +
> > +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > +	dev_dbg(cam->dev, "%s\n", __func__);
> > +}
> > +
> > +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> > +				   unsigned int *num_buffers,
> > +				   unsigned int *num_planes,
> > +				   unsigned int sizes[],
> > +				   struct device *alloc_devs[])
> > +{
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	unsigned int max_buffer_count = node->desc.max_buf_count;
> > +	const struct v4l2_format *fmt = &node->vdev_fmt;
> > +	unsigned int size;
> > +
> > +	/* Check the limitation of buffer size */
> > +	if (max_buffer_count)
> > +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> > +
> > +	if (node->desc.smem_alloc)
> > +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> > +
> > +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> > +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		size = fmt->fmt.meta.buffersize;
> > +	else
> > +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> > +
> > +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> > +	if (*num_planes) {
> > +		if (sizes[0] < size || *num_planes != 1)
> > +			return -EINVAL;
> > +	} else {
> > +		*num_planes = 1;
> > +		sizes[0] = size;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> > +					   struct mtk_cam_video_device *node,
> > +					   enum vb2_buffer_state state)
> > +{
> > +	struct mtk_cam_dev_buffer *buf, *buf_prev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&node->buf_list_lock, flags);
> > +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> > +		list_del(&buf->list);
> > +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
> > +	}
> > +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
> > +}
> > +
> > +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> > +				       unsigned int count)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!node->enabled) {
> > +		dev_err(dev, "Node:%d is not enabled\n", node->id);
> > +		ret = -ENOLINK;
> > +		goto fail_ret_buf;
> > +	}
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	/* Start streaming of the whole pipeline now*/
> > +	if (!cam->pipeline.streaming_count) {
> > +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> > +		if (ret) {
> > +			dev_err(dev, "failed to start pipeline:%d\n", ret);
> > +			goto fail_unlock;
> > +		}
> > +		mtk_cam_dev_init_stream(cam);
> > +		ret = mtk_isp_hw_init(cam);
> > +		if (ret) {
> > +			dev_err(dev, "failed to init HW:%d\n", ret);
> > +			goto fail_stop_pipeline;
> > +		}
> > +	}
> > +
> > +	/* Media links are fixed after media_pipeline_start */
> > +	cam->stream_count++;
> > +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> > +		cam->enabled_count);
> > +	if (cam->stream_count < cam->enabled_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return 0;
> > +	}
> > +
> > +	/* Stream on sub-devices node */
> > +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> > +	if (ret)
> > +		goto fail_no_stream;
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	return 0;
> > +
> > +fail_no_stream:
> > +	cam->stream_count--;
> > +fail_stop_pipeline:
> > +	if (cam->stream_count == 0)
> > +		media_pipeline_stop(&node->vdev.entity);
> > +fail_unlock:
> > +	mutex_unlock(&cam->op_lock);
> > +fail_ret_buf:
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> > +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> > +	struct device *dev = cam->dev;
> > +
> > +	mutex_lock(&cam->op_lock);
> > +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> > +		cam->stream_count);
> > +	/* Check the first node to stream-off */
> > +	if (cam->stream_count == cam->enabled_count)
> > +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> > +
> > +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> > +	cam->stream_count--;
> > +	if (cam->stream_count) {
> > +		mutex_unlock(&cam->op_lock);
> > +		return;
> > +	}
> > +	mutex_unlock(&cam->op_lock);
> > +
> > +	mtk_cam_dev_req_cleanup(cam);
> > +	media_pipeline_stop(&node->vdev.entity);
> > +}
> > +
> > +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> > +				   struct v4l2_capability *cap)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +
> > +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> > +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> > +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > +		 dev_name(cam->dev));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index >= node->desc.num_fmts)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt = node->vdev_fmt.fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> > +				  struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +	struct device *dev = cam->dev;
> > +	const struct v4l2_format *dev_fmt;
> > +	struct v4l2_format try_fmt;
> > +
> > +	memset(&try_fmt, 0, sizeof(try_fmt));
> > +	try_fmt.type = f->type;
> > +
> > +	/* Validate pixelformat */
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> > +	if (!dev_fmt) {
> > +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> > +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> > +	}
> > +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> > +
> > +	/* Validate image width & height range */
> > +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> > +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> > +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> > +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> > +	/* 4 bytes alignment for width */
> > +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> > +
> > +	/* Only support one plane */
> > +	try_fmt.fmt.pix_mp.num_planes = 1;
> > +
> > +	/* bytesperline & sizeimage calculation */
> > +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> > +
> > +	/* Constant format fields */
> > +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> > +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> > +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> > +
> > +	*f = try_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> > +				struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_dev *cam = video_drvdata(file);
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (vb2_is_busy(node->vdev.queue)) {
> > +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> > +		return -EBUSY;
> > +	}
> > +
> > +	/* Get the valid format */
> > +	mtk_cam_vidioc_try_fmt(file, fh, f);
> > +	/* Configure to video device */
> > +	node->vdev_fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> > +					  struct v4l2_frmsizeenum *sizes)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> > +	const struct v4l2_format *dev_fmt;
> > +
> > +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> > +	if (!dev_fmt || sizes->index)
> > +		return -EINVAL;
> > +
> > +	sizes->type = node->desc.frmsizes->type;
> > +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> > +	       sizeof(sizes->stepwise));
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> > +					struct v4l2_fmtdesc *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	if (f->index)
> > +		return -EINVAL;
> > +
> > +	/* f->description is filled in v4l_fill_fmtdesc function */
> > +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->flags = 0;
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> > +				     struct v4l2_format *f)
> > +{
> > +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> > +
> > +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> > +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> > +	.subscribe_event = mtk_cam_sd_subscribe_event,
> > +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> > +	.s_stream =  mtk_cam_sd_s_stream,
> > +};
> > +
> > +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> > +	.core = &mtk_cam_subdev_core_ops,
> > +	.video = &mtk_cam_subdev_video_ops,
> > +};
> > +
> > +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> > +	.link_setup = mtk_cam_media_link_setup,
> > +	.link_validate = v4l2_subdev_link_validate,
> > +};
> > +
> > +static const struct vb2_ops mtk_cam_vb2_ops = {
> > +	.queue_setup = mtk_cam_vb2_queue_setup,
> > +	.wait_prepare = vb2_ops_wait_prepare,
> > +	.wait_finish = vb2_ops_wait_finish,
> > +	.buf_init = mtk_cam_vb2_buf_init,
> > +	.buf_prepare = mtk_cam_vb2_buf_prepare,
> > +	.start_streaming = mtk_cam_vb2_start_streaming,
> > +	.stop_streaming = mtk_cam_vb2_stop_streaming,
> > +	.buf_queue = mtk_cam_vb2_buf_queue,
> > +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> > +	.buf_request_complete = mtk_cam_vb2_request_complete,
> > +};
> > +
> > +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> > +	.unlocked_ioctl = video_ioctl2,
> > +	.open = v4l2_fh_open,
> > +	.release = vb2_fop_release,
> > +	.poll = vb2_fop_poll,
> > +	.mmap = vb2_fop_mmap,
> > +#ifdef CONFIG_COMPAT
> > +	.compat_ioctl32 = v4l2_compat_ioctl32,
> > +#endif
> > +};
> > +
> > +static const struct media_device_ops mtk_cam_media_ops = {
> > +	.req_alloc = mtk_cam_req_alloc,
> > +	.req_free = mtk_cam_req_free,
> > +	.req_validate = vb2_request_validate,
> > +	.req_queue = mtk_cam_req_queue,
> > +};
> > +
> > +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> > +				  struct media_device *media_dev)
> > +{
> > +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> > +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	media_dev->dev = cam->dev;
> > +	strscpy(media_dev->model, dev_driver_string(dev),
> > +		sizeof(media_dev->model));
> > +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> > +		 "platform:%s", dev_name(dev));
> > +	media_dev->hw_revision = 0;
> > +	media_device_init(media_dev);
> > +	media_dev->ops = &mtk_cam_media_ops;
> > +
> > +	ret = media_device_register(media_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register media device:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/* Initialize subdev pads */
> > +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
> > +					sizeof(*cam->subdev_pads),
> > +					GFP_KERNEL);
> > +	if (!cam->subdev_pads) {
> > +		dev_err(dev, "failed to allocate subdev_pads\n");
> > +		ret = -ENOMEM;
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> > +				     cam->subdev_pads);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +
> > +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> > +	for (i = 0; i < num_pads; i++)
> > +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Customize the last one pad as CIO sink pad. */
> > +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +
> > +	return 0;
> > +
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> > +			      struct mtk_cam_video_device *node)
> > +{
> > +	struct device *dev = cam->dev;
> > +	struct video_device *vdev = &node->vdev;
> > +	struct vb2_queue *vbq = &node->vbq;
> > +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> > +	unsigned int link_flags = node->desc.link_flags;
> > +	int ret;
> > +
> > +	/* Initialize mtk_cam_video_device */
> > +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> > +		node->enabled = true;
> > +	else
> > +		node->enabled = false;
> > +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> > +
> > +	cam->subdev_pads[node->id].flags = output ?
> > +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > +
> > +	/* Initialize media entities */
> > +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
> > +		return ret;
> > +	}
> > +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > +
> > +	/* Initialize vbq */
> > +	vbq->type = node->desc.buf_type;
> > +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> > +		vbq->io_modes = VB2_MMAP;
> > +	else
> > +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> > +
> > +	if (node->desc.smem_alloc) {
> > +		vbq->bidirectional = 1;
> > +		vbq->dev = cam->smem_dev;
> > +	} else {
> > +		vbq->dev = dev;
> > +	}
> > +	vbq->ops = &mtk_cam_vb2_ops;
> > +	vbq->mem_ops = &vb2_dma_contig_memops;
> > +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> > +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> > +	if (output)
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> > +	else
> > +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> > +	/* No minimum buffers limitation */
> > +	vbq->min_buffers_needed = 0;
> > +	vbq->drv_priv = cam;
> > +	vbq->lock = &node->vdev_lock;
> > +	vbq->supports_requests = true;
> > +	vbq->requires_requests = true;
> > +
> > +	ret = vb2_queue_init(vbq);
> > +	if (ret) {
> > +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> > +		goto fail_media_clean;
> > +	}
> > +
> > +	/* Initialize vdev */
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +		 dev_driver_string(dev), node->desc.name);
> > +	/* set cap/type/ioctl_ops of the video device */
> > +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> > +	vdev->ioctl_ops = node->desc.ioctl_ops;
> > +	vdev->fops = &mtk_cam_v4l2_fops;
> > +	vdev->release = video_device_release_empty;
> > +	vdev->lock = &node->vdev_lock;
> > +	vdev->v4l2_dev = &cam->v4l2_dev;
> > +	vdev->queue = &node->vbq;
> > +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> > +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> > +	vdev->entity.ops = NULL;
> > +	video_set_drvdata(vdev, cam);
> > +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> > +
> > +	/* Initialize miscellaneous variables */
> > +	mutex_init(&node->vdev_lock);
> > +	INIT_LIST_HEAD(&node->buf_list);
> > +	spin_lock_init(&node->buf_list_lock);
> > +
> > +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register vde:%d\n", ret);
> > +		goto fail_vb2_rel;
> > +	}
> > +
> > +	/* Create link between video node and the subdev pad */
> > +	if (output) {
> > +		ret = media_create_pad_link(&vdev->entity, 0,
> > +					    &cam->subdev.entity,
> > +					    node->id, link_flags);
> > +	} else {
> > +		ret = media_create_pad_link(&cam->subdev.entity,
> > +					    node->id, &vdev->entity, 0,
> > +					    link_flags);
> > +	}
> > +	if (ret)
> > +		goto fail_vdev_ureg;
> > +
> > +	return 0;
> > +
> > +fail_vdev_ureg:
> > +	video_unregister_device(vdev);
> > +fail_vb2_rel:
> > +	mutex_destroy(&node->vdev_lock);
> > +	vb2_queue_release(vbq);
> > +fail_media_clean:
> > +	media_entity_cleanup(&vdev->entity);
> > +
> > +	return ret;
> > +}
> > +
> > +static void
> > +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> > +{
> > +	video_unregister_device(&node->vdev);
> > +	vb2_queue_release(&node->vbq);
> > +	media_entity_cleanup(&node->vdev.entity);
> > +	mutex_destroy(&node->vdev_lock);
> > +}
> > +
> > +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int i, ret;
> > +
> > +	/* Set up media device & pads */
> > +	ret = mtk_cam_media_register(cam, &cam->media_dev);
> > +	if (ret)
> > +		return ret;
> > +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> > +
> > +	/* Set up v4l2 device */
> > +	cam->v4l2_dev.mdev = &cam->media_dev;
> > +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> > +		goto fail_media_unreg;
> > +	}
> > +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> > +
> > +	/* Initialize subdev */
> > +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> > +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> > +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> > +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> > +				V4L2_SUBDEV_FL_HAS_EVENTS;
> > +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> > +		 "%s", dev_driver_string(dev));
> > +	v4l2_set_subdevdata(&cam->subdev, cam);
> > +
> > +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
> > +		goto fail_clean_media_entiy;
> > +	}
> > +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
> > +
> > +	/* Create video nodes and links */
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> > +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> > +
> > +		node->id = node->desc.id;
> > +		ret = mtk_cam_video_register_device(cam, node);
> > +		if (ret)
> > +			goto fail_vdev_unreg;
> > +	}
> > +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> > +
> > +	return 0;
> > +
> > +fail_vdev_unreg:
> > +	for (i--; i >= 0; i--)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +fail_clean_media_entiy:
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +fail_media_unreg:
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return ret;
> > +}
> > +
> > +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> > +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> > +
> > +	vb2_dma_contig_clear_max_seg_size(cam->dev);
> > +	v4l2_device_unregister_subdev(&cam->subdev);
> > +	v4l2_device_unregister(&cam->v4l2_dev);
> > +	media_entity_cleanup(&cam->subdev.entity);
> > +	media_device_unregister(&cam->media_dev);
> > +	media_device_cleanup(&cam->media_dev);
> > +
> > +	return 0;
> > +}
> > +
> > +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> > +				      struct v4l2_subdev *sd,
> > +				      struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> > +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	cam->seninf = sd;
> > +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> > +
> > +	return 0;
> > +}
> > +
> > +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> > +					struct v4l2_subdev *sd,
> > +					struct v4l2_async_subdev *asd)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +
> > +	cam->seninf = NULL;
> > +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> > +}
> > +
> > +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > +	struct mtk_cam_dev *cam =
> > +		container_of(notifier, struct mtk_cam_dev, notifier);
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	if (!cam->seninf) {
> > +		dev_err(dev, "No seninf subdev\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> > +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> > +				    MEDIA_LNK_FL_IMMUTABLE |
> > +				    MEDIA_LNK_FL_ENABLED);
> > +	if (ret) {
> > +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
> > +			cam->seninf->entity.name, cam->subdev.entity.name,
> > +			ret);
> > +		return ret;
> > +	}
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> > +	if (ret) {
> > +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> > +	.bound = mtk_cam_dev_notifier_bound,
> > +	.unbind = mtk_cam_dev_notifier_unbind,
> > +	.complete = mtk_cam_dev_notifier_complete,
> > +};
> > +
> > +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> > +{
> > +	struct device *dev = cam->dev;
> > +	int ret;
> > +
> > +	v4l2_async_notifier_init(&cam->notifier);
> > +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> > +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> > +	if (ret) {
> > +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> > +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> > +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> > +	if (ret) {
> > +		dev_err(dev, "failed to register async notifier : %d\n", ret);
> > +		v4l2_async_notifier_cleanup(&cam->notifier);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> > +{
> > +	v4l2_async_notifier_unregister(&cam->notifier);
> > +	v4l2_async_notifier_cleanup(&cam->notifier);
> > +}
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> > +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> > +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> > +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> > +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> > +	.vidioc_querycap = mtk_cam_vidioc_querycap,
> > +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> > +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> > +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
> > +	.vidioc_streamon = vb2_ioctl_streamon,
> > +	.vidioc_streamoff = vb2_ioctl_streamoff,
> > +	.vidioc_expbuf = vb2_ioctl_expbuf,
> > +};
> > +
> > +static const struct v4l2_format meta_fmts[] = {
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> > +			.buffersize = 512 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_3A,
> > +			.buffersize = 1200 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_AF,
> > +			.buffersize = 640 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LCS,
> > +			.buffersize = 288 * SZ_1K,
> > +		},
> > +	},
> > +	{
> > +		.fmt.meta = {
> > +			.dataformat = V4L2_META_FMT_MTISP_LMV,
> > +			.buffersize = 256,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format stream_out_fmts[] = {
> > +	/* This is a default image format */
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct v4l2_format bin_out_fmts[] = {
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> > +		},
> > +	},
> > +	{
> > +		.fmt.pix_mp = {
> > +			.width = IMG_MAX_WIDTH,
> > +			.height = IMG_MAX_HEIGHT,
> > +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> > +		},
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc output_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_META_IN_0,
> > +		.name = "meta input",
> > +		.cap = V4L2_CAP_META_OUTPUT,
> > +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = true,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 0,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> > +	},
> > +};
> > +
> > +static const struct
> > +mtk_cam_dev_node_desc capture_queues[] = {
> > +	{
> > +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> > +		.name = "main stream",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_IMGO,
> > +		.fmts = stream_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
> > +		.name = "packed out",
> > +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> > +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> > +		.link_flags = 0,
> > +		.image = true,
> > +		.smem_alloc = false,
> > +		.dma_port = R_RRZO,
> > +		.fmts = bin_out_fmts,
> > +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
> > +		.default_fmt_idx = 0,
> > +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> > +		.frmsizes = &(struct v4l2_frmsizeenum) {
> > +			.index = 0,
> > +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> > +			.stepwise = {
> > +				.max_width = IMG_MAX_WIDTH,
> > +				.min_width = IMG_MIN_WIDTH,
> > +				.max_height = IMG_MAX_HEIGHT,
> > +				.min_height = IMG_MIN_HEIGHT,
> > +				.step_height = 1,
> > +				.step_width = 1,
> > +			},
> > +		},
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_0,
> > +		.name = "partial meta 0",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AAO | R_FLKO | R_PSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 1,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_1,
> > +		.name = "partial meta 1",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_AFO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 2,
> > +		.max_buf_count = 5,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_2,
> > +		.name = "partial meta 2",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LCSO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 3,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +	{
> > +		.id = MTK_CAM_P1_META_OUT_3,
> > +		.name = "partial meta 3",
> > +		.cap = V4L2_CAP_META_CAPTURE,
> > +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> > +		.link_flags = 0,
> > +		.image = false,
> > +		.smem_alloc = false,
> > +		.dma_port = R_LMVO,
> > +		.fmts = meta_fmts,
> > +		.default_fmt_idx = 4,
> > +		.max_buf_count = 10,
> > +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> > +	},
> > +};
> > +
> > +/* The helper to configure the device context */
> > +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> > +{
> > +	unsigned int node_idx;
> > +	int i;
> > +
> > +	node_idx = 0;
> > +	/* Setup the output queue */
> > +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
> > +
> > +	/* Setup the capture queue */
> > +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> > +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> > +}
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam)
> > +{
> > +	int ret;
> > +
> > +	cam->dev = &pdev->dev;
> > +	mtk_cam_dev_queue_setup(cam);
> > +
> > +	spin_lock_init(&cam->pending_job_lock);
> > +	spin_lock_init(&cam->running_job_lock);
> > +	INIT_LIST_HEAD(&cam->pending_job_list);
> > +	INIT_LIST_HEAD(&cam->running_job_list);
> > +	mutex_init(&cam->op_lock);
> > +
> > +	/* v4l2 sub-device registration */
> > +	ret = mtk_cam_v4l2_register(cam);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = mtk_cam_v4l2_async_register(cam);
> > +	if (ret)
> > +		goto fail_v4l2_unreg;
> > +
> > +	return 0;
> > +
> > +fail_v4l2_unreg:
> > +	mutex_destroy(&cam->op_lock);
> > +	mtk_cam_v4l2_unregister(cam);
> > +
> > +	return ret;
> > +}
> > +
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> > +{
> > +	mtk_cam_v4l2_async_unregister(cam);
> > +	mtk_cam_v4l2_unregister(cam);
> > +	mutex_destroy(&cam->op_lock);
> > +}
> > +
> > diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > new file mode 100644
> > index 000000000000..0a340a1e65ea
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > @@ -0,0 +1,244 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c) 2019 MediaTek Inc.
> > + */
> > +
> > +#ifndef __MTK_CAM_H__
> > +#define __MTK_CAM_H__
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/videodev2.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-subdev.h>
> > +#include <media/videobuf2-core.h>
> > +#include <media/videobuf2-v4l2.h>
> > +
> > +#include "mtk_cam-ipi.h"
> > +
> > +#define IMG_MAX_WIDTH		5376
> > +#define IMG_MAX_HEIGHT		4032
> > +#define IMG_MIN_WIDTH		80
> > +#define IMG_MIN_HEIGHT		60
> > +
> > +/*
> > + * ID enum value for struct mtk_cam_dev_node_desc:id
> > + * or mtk_cam_video_device:id
> > + */
> > +enum  {
> > +	MTK_CAM_P1_META_IN_0 = 0,
> > +	MTK_CAM_P1_MAIN_STREAM_OUT,
> > +	MTK_CAM_P1_PACKED_BIN_OUT,
> > +	MTK_CAM_P1_META_OUT_0,
> > +	MTK_CAM_P1_META_OUT_1,
> > +	MTK_CAM_P1_META_OUT_2,
> > +	MTK_CAM_P1_META_OUT_3,
> > +	MTK_CAM_P1_TOTAL_NODES
> > +};
> > +
> > +/* Supported image format list */
> > +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
> > +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
> > +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
> > +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
> > +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
> > +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
> > +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
> > +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
> > +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
> > +
> > +/* Supported bayer pixel order */
> > +#define MTK_CAM_RAW_PXL_ID_B		0
> > +#define MTK_CAM_RAW_PXL_ID_GB		1
> > +#define MTK_CAM_RAW_PXL_ID_GR		2
> > +#define MTK_CAM_RAW_PXL_ID_R		3
> > +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
> > +
> > +/*
> > + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> > + *
> > + * @frame_seq_no: The frame sequence of frame in driver layer.
> > + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> > + *
> > + */
> > +struct mtk_p1_frame_param {
> > +	unsigned int frame_seq_no;
> > +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> > +} __packed;
> > +
> > +/*
> > + * struct mtk_cam_dev_request - MTK camera device request.
> > + *
> > + * @req: Embedded struct media request.
> > + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> > + * @frame_work: work queue entry for frame transmission to SCP.
> > + * @list: List entry of the object for @struct mtk_cam_dev:
> > + *        pending_job_list or running_job_list.
> > + * @timestamp: Start of frame timestamp in ns
> > + *
> > + */
> > +struct mtk_cam_dev_request {
> > +	struct media_request req;
> > +	struct mtk_p1_frame_param frame_params;
> > +	struct work_struct frame_work;
> > +	struct list_head list;
> > +	u64 timestamp;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> > + *
> > + * @vbb: Embedded struct vb2_v4l2_buffer.
> > + * @list: List entry of the object for @struct mtk_cam_video_device:
> > + *        buf_list.
> > + * @daddr: The DMA address of this buffer.
> > + * @scp_addr: The SCP address of this buffer which
> > + *            is only supported for meta input node.
> > + * @node_id: The vidoe node id which this buffer belongs to.
> > + *
> > + */
> > +struct mtk_cam_dev_buffer {
> > +	struct vb2_v4l2_buffer vbb;
> > +	struct list_head list;
> > +	/* Intenal part */
> > +	dma_addr_t daddr;
> > +	dma_addr_t scp_addr;
> > +	unsigned int node_id;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> > + *
> > + * @id: id of the node
> > + * @name: name of the node
> > + * @cap: supported V4L2 capabilities
> > + * @buf_type: supported V4L2 buffer type
> > + * @dma_port: the dma ports associated to the node
> > + * @link_flags: default media link flags
> > + * @smem_alloc: using the smem_dev as alloc device or not
> > + * @image: true for image node, false for meta node
> > + * @num_fmts: the number of supported node formats
> > + * @default_fmt_idx: default format of this node
> > + * @max_buf_count: maximum VB2 buffer count
> > + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> > + * @fmts: supported format
> > + * @frmsizes: supported V4L2 frame size number
> > + *
> > + */
> > +struct mtk_cam_dev_node_desc {
> > +	u8 id;
> > +	const char *name;
> > +	u32 cap;
> > +	u32 buf_type;
> > +	u32 dma_port;
> > +	u32 link_flags;
> > +	u8 smem_alloc:1;
> > +	u8 image:1;
> > +	u8 num_fmts;
> > +	u8 default_fmt_idx;
> > +	u8 max_buf_count;
> > +	const struct v4l2_ioctl_ops *ioctl_ops;
> > +	const struct v4l2_format *fmts;
> > +	const struct v4l2_frmsizeenum *frmsizes;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_video_device - Mediatek video device structure
> > + *
> > + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> > + * @enabled: Indicate the video device is enabled or not
> > + * @desc: The node description of video device
> > + * @vdev_fmt: The V4L2 format of video device
> > + * @vdev_pad: The media pad graph object of video device
> > + * @vbq: A videobuf queue of video device
> > + * @vdev: The video device instance
> > + * @vdev_lock: Serializes vb2 queue and video device operations
> > + * @buf_list: List for enqueue buffers
> > + * @buf_list_lock: Lock used to protect buffer list.
> > + *
> > + */
> > +struct mtk_cam_video_device {
> > +	unsigned int id;
> > +	unsigned int enabled;
> > +	struct mtk_cam_dev_node_desc desc;
> > +	struct v4l2_format vdev_fmt;
> > +	struct media_pad vdev_pad;
> > +	struct vb2_queue vbq;
> > +	struct video_device vdev;
> > +	/* Serializes vb2 queue and video device operations */
> > +	struct mutex vdev_lock;
> > +	struct list_head buf_list;
> > +	/* Lock used to protect buffer list */
> > +	spinlock_t buf_list_lock;
> > +};
> > +
> > +/*
> > + * struct mtk_cam_dev - Mediatek camera device structure.
> > + *
> > + * @dev: Pointer to device.
> > + * @smem_pdev: Pointer to shared memory device.
> > + * @pipeline: Media pipeline information.
> > + * @media_dev: Media device instance.
> > + * @subdev: The V4L2 sub-device instance.
> > + * @v4l2_dev: The V4L2 device driver instance.
> > + * @notifier: The v4l2_device notifier data.
> > + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> > + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> > + * @seninf: Pointer to the seninf sub-device.
> > + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> > + * @streaming: Indicate the overall streaming status is on or off.
> > + * @enabled_dmas: The enabled dma port information when streaming on.
> > + * @enabled_count: Number of enabled video nodes
> > + * @stream_count: Number of streaming video nodes
> > + * @running_job_count: Nunber of running jobs in the HW driver.
> > + * @pending_job_list: List to keep the media requests before en-queue into
> > + *                    HW driver.
> > + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> > + * @running_job_list: List to keep the media requests after en-queue into
> > + *                    HW driver.
> > + * @running_job_lock: Protect the running_job_list data.
> > + * @op_lock: Serializes driver's VB2 callback operations.
> > + *
> > + */
> > +struct mtk_cam_dev {
> > +	struct device *dev;
> > +	struct device *smem_dev;
> > +	struct media_pipeline pipeline;
> > +	struct media_device media_dev;
> > +	struct v4l2_subdev subdev;
> > +	struct v4l2_device v4l2_dev;
> > +	struct v4l2_async_notifier notifier;
> > +	struct media_pad *subdev_pads;
> > +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> > +	struct v4l2_subdev *seninf;
> > +	struct v4l2_subdev *sensor;
> > +	unsigned int streaming;
> > +	unsigned int enabled_dmas;
> > +	unsigned int enabled_count;
> > +	unsigned int stream_count;
> > +	unsigned int running_job_count;
> > +	struct list_head pending_job_list;
> > +	/* Protect the pending_job_list data */
> > +	spinlock_t pending_job_lock;
> > +	struct list_head running_job_list;
> > +	/* Protect the running_job_list data & running_job_count */
> > +	spinlock_t running_job_lock;
> > +	/* Serializes driver's VB2 callback operations */
> > +	struct mutex op_lock;
> > +};
> > +
> > +int mtk_cam_dev_init(struct platform_device *pdev,
> > +		     struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> > +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> > +				   unsigned int frame_seq_no);
> > +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> > +				  unsigned int frame_seq_no);
> > +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> > +						unsigned int frame_seq_no);
> > +
> > +#endif /* __MTK_CAM_H__ */
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
  2020-04-03  2:30       ` Laurent Pinchart
  (?)
@ 2020-04-10 10:00         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:00 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: tfiga, hverkuil-cisco, matthias.bgg, mchehab, linux-media,
	linux-mediatek, linux-arm-kernel, devicetree, srv_heupstream,
	ddavenport, robh, Sean.Cheng, sj.huang, frederic.chen,
	Jerry-ch.Chen, frankie.chiu, ryan.yu, Rynn.Wu, yuzhao, zwisler,
	shik, suleiman

Hi, Laurent:

Thanks for your comments.

On Fri, 2020-04-03 at 05:30 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> > Add packed/full-g bayer formats with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> > 
> > (The current metadata format used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
> >    v5.5-rc1 version.
> > ---
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
> >  8 files changed, 638 insertions(+)
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > 
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > new file mode 100644
> > index 000000000000..534edb4f0fd4
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > @@ -0,0 +1,65 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> > +.. _v4l2-pix-fmt-mtisp-srggb10:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> > +*******************************
> > +
> > +10-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +    * - start + 2:
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - G\ :sub:`03high bits 9--2`
> 
> This contradicts the description above, where you mention there's no
> padding, and here only 8 bits are used for the two bytes. Which one is
> correct ?
> 

These four pixel formats are raw sRGB / Bayer formats with 10 bits per
color. Each color component is stored in the 1st byte with bit 0~7, with
2 extra high bits 8~9 will be stored in 2nd byte. For the rest 6 bits of
2nd byte are filled with the next color with bit 0~5. So there is no
padding between the consecutive colors.

> > +    * - start + 6:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +    * - start + 8:
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 10:
> > +      - R\ :sub:`13high bits 9--2`
> > +    * - start + 12:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +    * - start + 14:
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 16:
> > +      - G\ :sub:`23high bits 9--2`
> > +    * - start + 18:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +    * - start + 20:
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 22:
> > +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > new file mode 100644
> > index 000000000000..7be527711602
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > @@ -0,0 +1,90 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> > +.. _v4l2-pix-fmt-mtisp-srggb10f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> > +*******************************
> > +
> > +10-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> 
> I think this should describe where the additional green pixel comes
> from.
> 

Ok, we will add more descriptions to describe.

The Full-G format adopts some of the features of Bayer CFA and RGB.
In R and B Channels, only the pixel value of the corresponding position
under the CFA arrangement is recorded.
And the G Channel has full pixel values.


> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - B\ :sub:`03high bits 9--2`
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`05high bits 3--0`
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 12:
> > +      - G\ :sub:`13high bits 9--2`
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`15high bits 3--0`
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 20:
> > +      - B\ :sub:`23high bits 9--2`
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`25high bits 3--0`
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 28:
> > +      - G\ :sub:`33high bits 9--2`
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`35high bits 3--0`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > new file mode 100644
> > index 000000000000..cc888aac42c2
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > @@ -0,0 +1,61 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> > +.. _v4l2-pix-fmt-mtisp-srggb12:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> > +*******************************
> > +
> > +12-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00lowbits 7--0`
> > +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`01highbits 7--0`
> > +      - B\ :sub:`02lowbits 7--0`
> > +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`03highbits 7--0`
> > +    * - start + 6:
> > +      - G\ :sub:`10lowbits 7--0`
> > +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`11highbits 7--0`
> > +      - G\ :sub:`12lowbits 7--0`
> > +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`13highbits 7--0`
> > +    * - start + 12:
> > +      - B\ :sub:`20lowbits 7--0`
> > +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`21highbits 7--0`
> > +      - B\ :sub:`22lowbits 7--0`
> > +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`23highbits 7--0`
> > +    * - start + 18:
> > +      - G\ :sub:`30lowbits 7--0`
> > +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`31highbits 7--0`
> > +      - G\ :sub:`32lowbits 7--0`
> > +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`33highbits 7--0`
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > new file mode 100644
> > index 000000000000..c063de9f9ad8
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> > +.. _v4l2-pix-fmt-mtisp-srggb12f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> > +*******************************
> > +
> > +12-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> > +    * - start + 2:
> > +      - FG\ :sub:`01high bits 7--0`
> > +      - G\ :sub:`02low bits 7--0`
> > +    * - start + 4:
> > +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`03high bits 7--0`
> > +    * - start + 6:
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> > +    * - start + 8:
> > +      - G\ :sub:`05high bits 7--0`
> > +      -
> > +    * - start + 10:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> > +    * - start + 12:
> > +      - R\ :sub:`11high bits 7--0`
> > +      - FG\ :sub:`12low bits 7--0`
> > +    * - start + 14:
> > +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`13high bits 7--0`
> > +    * - start + 16:
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> > +    * - start + 18:
> > +      - FG\ :sub:`15high bits 7--0`
> > +      -
> > +    * - start + 20:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> > +    * - start + 22:
> > +      - FG\ :sub:`21high bits 7--0`
> > +      - G\ :sub:`22low bits 7--0`
> > +    * - start + 24:
> > +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`23high bits 7--0`
> > +    * - start + 26:
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`25high bits 7--0`
> > +      -
> > +    * - start + 30:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> > +    * - start + 32:
> > +      - R\ :sub:`31high bits 7--0`
> > +      - FG\ :sub:`32low bits 7--0`
> > +    * - start + 34:
> > +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`33high bits 7--0`
> > +    * - start + 36:
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> > +    * - start + 38:
> > +      - FG\ :sub:`35high bits 7--0`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > new file mode 100644
> > index 000000000000..39ea9882a792
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > @@ -0,0 +1,73 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> > +.. _v4l2-pix-fmt-mtisp-srggb14:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> > +*******************************
> > +
> > +14-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`01low bits 9--2`\
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - B\ :sub:`02low bits 11--4`\
> > +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`03high bits 13--6`\
> > +      -
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`\
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 12:
> > +      - G\ :sub:`12low bits 11--4`\
> > +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`13high bits 13--6`\
> > +      -
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`21low bits 9--2`\
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 20:
> > +      - B\ :sub:`22low bits 11--4`\
> > +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`23high bits 13--6`\
> > +      -
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`\
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`32low bits 11--4`\
> > +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`33high bits 13--6`\
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > new file mode 100644
> > index 000000000000..010b1c190c60
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> > +.. _v4l2-pix-fmt-mtisp-srggb14f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> > +*******************************
> > +
> > +14-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`01low bits 9--2`
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - G\ :sub:`02low bits 11--4`
> > +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`03high bits 13--6`
> > +      - FG\ :sub:`04low bits 7--0`
> > +    * - start + 8:
> > +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`05high bits 9--2`
> > +      - G\ :sub:`05high bits 13--10`
> > +      -
> > +    * - start + 12:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 16:
> > +      - FG\ :sub:`12low bits 11--4`
> > +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`13high bits 13--6`
> > +      - R\ :sub:`14low bits 7--0`
> > +    * - start + 20:
> > +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`15high bits 9--2`
> > +      - FG\ :sub:`15high bits 13--10`
> > +      -
> > +    * - start + 24:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`21low bits 9--2`
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`22low bits 11--4`
> > +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`23high bits 13--6`
> > +      - FG\ :sub:`24low bits 7--0`
> > +    * - start + 32:
> > +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`25high bits 9--2`
> > +      - G\ :sub:`25high bits 13--10`
> > +      -
> > +    * - start + 36:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 40:
> > +      - FG\ :sub:`32low bits 11--4`
> > +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`33high bits 13--6`
> > +      - R\ :sub:`34low bits 7--0`
> > +    * - start + 44:
> > +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`35high bits 9--2`
> > +      - FG\ :sub:`35high bits 13--10`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > new file mode 100644
> > index 000000000000..86cadbf38175
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > @@ -0,0 +1,51 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> > +.. _v4l2-pix-fmt-mtisp-srggb8:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> > +*******************************
> > +
> > +8-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> 
> How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
> other variants) ? They seem identical based on the description.
> 

You are right. They are identical.
We will move "8-bit packed bayer formats" in next patch.

Best regards,

Jungo

> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - G\ :sub:`01`
> > +      - B\ :sub:`02`
> > +      - G\ :sub:`03`
> > +    * - start + 4:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - G\ :sub:`12`
> > +      - R\ :sub:`13`
> > +    * - start + 8:
> > +      - B\ :sub:`20`
> > +      - G\ :sub:`21`
> > +      - B\ :sub:`22`
> > +      - G\ :sub:`23`
> > +    * - start + 12:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - G\ :sub:`32`
> > +      - R\ :sub:`33`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > new file mode 100644
> > index 000000000000..ca5151312bca
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > @@ -0,0 +1,78 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> > +.. _v4l2-pix-fmt-mtisp-srggb8f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> > +*******************************
> > +
> > +8-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - start + 6:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +    * - start + 12:
> > +      - B\ :sub:`20`
> > +      - FG\ :sub:`21`
> > +      - G\ :sub:`22`
> > +      - B\ :sub:`23`
> > +      - FG\ :sub:`24`
> > +      - G\ :sub:`25`
> > +    * - start + 18:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - FG\ :sub:`32`
> > +      - G\ :sub:`33`
> > +      - R\ :sub:`34`
> > +      - FG\ :sub:`35`
> > \ No newline at end of file
> 


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2020-04-10 10:00         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:00 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: ryan.yu, frankie.chiu, robh, Rynn.Wu, suleiman, Jerry-ch.Chen,
	frederic.chen, linux-media, devicetree, hverkuil-cisco, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi, Laurent:

Thanks for your comments.

On Fri, 2020-04-03 at 05:30 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> > Add packed/full-g bayer formats with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> > 
> > (The current metadata format used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
> >    v5.5-rc1 version.
> > ---
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
> >  8 files changed, 638 insertions(+)
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > 
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > new file mode 100644
> > index 000000000000..534edb4f0fd4
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > @@ -0,0 +1,65 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> > +.. _v4l2-pix-fmt-mtisp-srggb10:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> > +*******************************
> > +
> > +10-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +    * - start + 2:
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - G\ :sub:`03high bits 9--2`
> 
> This contradicts the description above, where you mention there's no
> padding, and here only 8 bits are used for the two bytes. Which one is
> correct ?
> 

These four pixel formats are raw sRGB / Bayer formats with 10 bits per
color. Each color component is stored in the 1st byte with bit 0~7, with
2 extra high bits 8~9 will be stored in 2nd byte. For the rest 6 bits of
2nd byte are filled with the next color with bit 0~5. So there is no
padding between the consecutive colors.

> > +    * - start + 6:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +    * - start + 8:
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 10:
> > +      - R\ :sub:`13high bits 9--2`
> > +    * - start + 12:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +    * - start + 14:
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 16:
> > +      - G\ :sub:`23high bits 9--2`
> > +    * - start + 18:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +    * - start + 20:
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 22:
> > +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > new file mode 100644
> > index 000000000000..7be527711602
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > @@ -0,0 +1,90 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> > +.. _v4l2-pix-fmt-mtisp-srggb10f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> > +*******************************
> > +
> > +10-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> 
> I think this should describe where the additional green pixel comes
> from.
> 

Ok, we will add more descriptions to describe.

The Full-G format adopts some of the features of Bayer CFA and RGB.
In R and B Channels, only the pixel value of the corresponding position
under the CFA arrangement is recorded.
And the G Channel has full pixel values.


> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - B\ :sub:`03high bits 9--2`
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`05high bits 3--0`
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 12:
> > +      - G\ :sub:`13high bits 9--2`
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`15high bits 3--0`
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 20:
> > +      - B\ :sub:`23high bits 9--2`
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`25high bits 3--0`
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 28:
> > +      - G\ :sub:`33high bits 9--2`
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`35high bits 3--0`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > new file mode 100644
> > index 000000000000..cc888aac42c2
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > @@ -0,0 +1,61 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> > +.. _v4l2-pix-fmt-mtisp-srggb12:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> > +*******************************
> > +
> > +12-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00lowbits 7--0`
> > +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`01highbits 7--0`
> > +      - B\ :sub:`02lowbits 7--0`
> > +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`03highbits 7--0`
> > +    * - start + 6:
> > +      - G\ :sub:`10lowbits 7--0`
> > +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`11highbits 7--0`
> > +      - G\ :sub:`12lowbits 7--0`
> > +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`13highbits 7--0`
> > +    * - start + 12:
> > +      - B\ :sub:`20lowbits 7--0`
> > +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`21highbits 7--0`
> > +      - B\ :sub:`22lowbits 7--0`
> > +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`23highbits 7--0`
> > +    * - start + 18:
> > +      - G\ :sub:`30lowbits 7--0`
> > +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`31highbits 7--0`
> > +      - G\ :sub:`32lowbits 7--0`
> > +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`33highbits 7--0`
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > new file mode 100644
> > index 000000000000..c063de9f9ad8
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> > +.. _v4l2-pix-fmt-mtisp-srggb12f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> > +*******************************
> > +
> > +12-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> > +    * - start + 2:
> > +      - FG\ :sub:`01high bits 7--0`
> > +      - G\ :sub:`02low bits 7--0`
> > +    * - start + 4:
> > +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`03high bits 7--0`
> > +    * - start + 6:
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> > +    * - start + 8:
> > +      - G\ :sub:`05high bits 7--0`
> > +      -
> > +    * - start + 10:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> > +    * - start + 12:
> > +      - R\ :sub:`11high bits 7--0`
> > +      - FG\ :sub:`12low bits 7--0`
> > +    * - start + 14:
> > +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`13high bits 7--0`
> > +    * - start + 16:
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> > +    * - start + 18:
> > +      - FG\ :sub:`15high bits 7--0`
> > +      -
> > +    * - start + 20:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> > +    * - start + 22:
> > +      - FG\ :sub:`21high bits 7--0`
> > +      - G\ :sub:`22low bits 7--0`
> > +    * - start + 24:
> > +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`23high bits 7--0`
> > +    * - start + 26:
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`25high bits 7--0`
> > +      -
> > +    * - start + 30:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> > +    * - start + 32:
> > +      - R\ :sub:`31high bits 7--0`
> > +      - FG\ :sub:`32low bits 7--0`
> > +    * - start + 34:
> > +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`33high bits 7--0`
> > +    * - start + 36:
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> > +    * - start + 38:
> > +      - FG\ :sub:`35high bits 7--0`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > new file mode 100644
> > index 000000000000..39ea9882a792
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > @@ -0,0 +1,73 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> > +.. _v4l2-pix-fmt-mtisp-srggb14:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> > +*******************************
> > +
> > +14-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`01low bits 9--2`\
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - B\ :sub:`02low bits 11--4`\
> > +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`03high bits 13--6`\
> > +      -
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`\
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 12:
> > +      - G\ :sub:`12low bits 11--4`\
> > +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`13high bits 13--6`\
> > +      -
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`21low bits 9--2`\
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 20:
> > +      - B\ :sub:`22low bits 11--4`\
> > +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`23high bits 13--6`\
> > +      -
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`\
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`32low bits 11--4`\
> > +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`33high bits 13--6`\
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > new file mode 100644
> > index 000000000000..010b1c190c60
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> > +.. _v4l2-pix-fmt-mtisp-srggb14f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> > +*******************************
> > +
> > +14-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`01low bits 9--2`
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - G\ :sub:`02low bits 11--4`
> > +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`03high bits 13--6`
> > +      - FG\ :sub:`04low bits 7--0`
> > +    * - start + 8:
> > +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`05high bits 9--2`
> > +      - G\ :sub:`05high bits 13--10`
> > +      -
> > +    * - start + 12:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 16:
> > +      - FG\ :sub:`12low bits 11--4`
> > +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`13high bits 13--6`
> > +      - R\ :sub:`14low bits 7--0`
> > +    * - start + 20:
> > +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`15high bits 9--2`
> > +      - FG\ :sub:`15high bits 13--10`
> > +      -
> > +    * - start + 24:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`21low bits 9--2`
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`22low bits 11--4`
> > +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`23high bits 13--6`
> > +      - FG\ :sub:`24low bits 7--0`
> > +    * - start + 32:
> > +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`25high bits 9--2`
> > +      - G\ :sub:`25high bits 13--10`
> > +      -
> > +    * - start + 36:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 40:
> > +      - FG\ :sub:`32low bits 11--4`
> > +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`33high bits 13--6`
> > +      - R\ :sub:`34low bits 7--0`
> > +    * - start + 44:
> > +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`35high bits 9--2`
> > +      - FG\ :sub:`35high bits 13--10`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > new file mode 100644
> > index 000000000000..86cadbf38175
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > @@ -0,0 +1,51 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> > +.. _v4l2-pix-fmt-mtisp-srggb8:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> > +*******************************
> > +
> > +8-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> 
> How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
> other variants) ? They seem identical based on the description.
> 

You are right. They are identical.
We will move "8-bit packed bayer formats" in next patch.

Best regards,

Jungo

> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - G\ :sub:`01`
> > +      - B\ :sub:`02`
> > +      - G\ :sub:`03`
> > +    * - start + 4:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - G\ :sub:`12`
> > +      - R\ :sub:`13`
> > +    * - start + 8:
> > +      - B\ :sub:`20`
> > +      - G\ :sub:`21`
> > +      - B\ :sub:`22`
> > +      - G\ :sub:`23`
> > +    * - start + 12:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - G\ :sub:`32`
> > +      - R\ :sub:`33`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > new file mode 100644
> > index 000000000000..ca5151312bca
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > @@ -0,0 +1,78 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> > +.. _v4l2-pix-fmt-mtisp-srggb8f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> > +*******************************
> > +
> > +8-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - start + 6:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +    * - start + 12:
> > +      - B\ :sub:`20`
> > +      - FG\ :sub:`21`
> > +      - G\ :sub:`22`
> > +      - B\ :sub:`23`
> > +      - FG\ :sub:`24`
> > +      - G\ :sub:`25`
> > +    * - start + 18:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - FG\ :sub:`32`
> > +      - G\ :sub:`33`
> > +      - R\ :sub:`34`
> > +      - FG\ :sub:`35`
> > \ No newline at end of file
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats
@ 2020-04-10 10:00         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:00 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: ryan.yu, frankie.chiu, robh, Rynn.Wu, suleiman, Jerry-ch.Chen,
	frederic.chen, linux-media, devicetree, hverkuil-cisco, shik,
	yuzhao, linux-mediatek, matthias.bgg, mchehab, linux-arm-kernel,
	Sean.Cheng, srv_heupstream, sj.huang, tfiga, zwisler, ddavenport

Hi, Laurent:

Thanks for your comments.

On Fri, 2020-04-03 at 05:30 +0300, Laurent Pinchart wrote:
> Hi Jungo,
> 
> Thank you for the patch.
> 
> On Thu, Dec 19, 2019 at 01:49:29PM +0800, Jungo Lin wrote:
> > Add packed/full-g bayer formats with 8/10/12/14 bit
> > for image output. Add Pass 1 (P1) specific meta formats for
> > parameter processing and 3A/other statistics.
> > 
> > (The current metadata format used in meta input and partial
> > meta nodes is only a temporary solution to kick off the driver
> > development and is not ready to be reviewed yet.)
> > 
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Remove RGB format definitions in pixfmt-rgb.rst for kernel
> >    v5.5-rc1 version.
> > ---
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |  65 +++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |  90 ++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |  61 ++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |  73 ++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  | 110 ++++++++++++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |  51 ++++++++
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |  78 +++++++++++++
> >  8 files changed, 638 insertions(+)
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > 
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > new file mode 100644
> > index 000000000000..534edb4f0fd4
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> > @@ -0,0 +1,65 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10:
> > +.. _v4l2-pix-fmt-mtisp-srggb10:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10 ('MBBA'), V4L2_PIX_FMT_MTISP_SGBRG10('MBGA'), V4L2_PIX_FMT_MTISP_SGRBG10('MBgA'), V4L2_PIX_FMT_MTISP_SRGGB10('MBRA')
> > +*******************************
> > +
> > +10-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 10 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 5 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 5--0` (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +    * - start + 2:
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`03low bits 1--0`\ (bits 7--6) B\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - G\ :sub:`03high bits 9--2`
> 
> This contradicts the description above, where you mention there's no
> padding, and here only 8 bits are used for the two bytes. Which one is
> correct ?
> 

These four pixel formats are raw sRGB / Bayer formats with 10 bits per
color. Each color component is stored in the 1st byte with bit 0~7, with
2 extra high bits 8~9 will be stored in 2nd byte. For the rest 6 bits of
2nd byte are filled with the next color with bit 0~5. So there is no
padding between the consecutive colors.

> > +    * - start + 6:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +    * - start + 8:
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`13low bits 1--0`\ (bits 7--6) G\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 10:
> > +      - R\ :sub:`13high bits 9--2`
> > +    * - start + 12:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +    * - start + 14:
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`23low bits 1--0`\ (bits 7--6) B\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 16:
> > +      - G\ :sub:`23high bits 9--2`
> > +    * - start + 18:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +    * - start + 20:
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - R\ :sub:`33low bits 1--0`\ (bits 7--6) G\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 22:
> > +      - R\ :sub:`33high bits 9--2` (bits 7--0)
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > new file mode 100644
> > index 000000000000..7be527711602
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> > @@ -0,0 +1,90 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr10f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg10f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg10f:
> > +.. _v4l2-pix-fmt-mtisp-srggb10f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR10F ('MFBA'), V4L2_PIX_FMT_MTISP_SGBRG10F('MFGA'), V4L2_PIX_FMT_MTISP_SGRBG10F('MFgA'), V4L2_PIX_FMT_MTISP_SRGGB10F('MFRA')
> > +*******************************
> > +
> > +10-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 10 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> 
> I think this should describe where the additional green pixel comes
> from.
> 

Ok, we will add more descriptions to describe.

The Full-G format adopts some of the features of Bayer CFA and RGB.
In R and B Channels, only the pixel value of the corresponding position
under the CFA arrangement is recorded.
And the G Channel has full pixel values.


> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 5--0`\ (bits 7--2) B\ :sub:`00high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`03low bits 1--0`\ (bits 7--6) G\ :sub:`02high bits 9--4`\ (bits 5--0)
> > +    * - start + 4:
> > +      - B\ :sub:`03high bits 9--2`
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 5--0`\ (bits 7--2) FG\ :sub:`04high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`05high bits 3--0`
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 5--0`\ (bits 7--2) G\ :sub:`10high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`13low bits 1--0`\ (bits 7--6) FG\ :sub:`12high bits 9--4`\ (bits 5--0)
> > +    * - start + 12:
> > +      - G\ :sub:`13high bits 9--2`
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 5--0`\ (bits 7--2) R\ :sub:`14high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`15high bits 3--0`
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 5--0`\ (bits 7--2) B\ :sub:`20high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 9--6`\ (bits 3--0)
> > +      - B\ :sub:`23low bits 1--0`\ (bits 7--6) G\ :sub:`22high bits 9--4`\ (bits 5--0)
> > +    * - start + 20:
> > +      - B\ :sub:`23high bits 9--2`
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 5--0`\ (bits 7--2) FG\ :sub:`24high bits 9--8`\ (bits 1--0)
> > +      - G\ :sub:`25high bits 3--0`
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 5--0`\ (bits 7--2) G\ :sub:`30high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 9--6`\ (bits 3--0)
> > +      - G\ :sub:`33low bits 1--0`\ (bits 7--6) FG\ :sub:`32high bits 9--4`\ (bits 5--0)
> > +    * - start + 28:
> > +      - G\ :sub:`33high bits 9--2`
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 5--0`\ (bits 7--2) R\ :sub:`34high bits 9--8`\ (bits 1--0)
> > +      - FG\ :sub:`35high bits 3--0`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > new file mode 100644
> > index 000000000000..cc888aac42c2
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> > @@ -0,0 +1,61 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12:
> > +.. _v4l2-pix-fmt-mtisp-srggb12:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12 ('MBBC'), V4L2_PIX_FMT_MTISP_SGBRG12('MBGC'), V4L2_PIX_FMT_MTISP_SGRBG12('MBgC'), V4L2_PIX_FMT_MTISP_SRGGB12('MBRC')
> > +*******************************
> > +
> > +12-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 12 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 6 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00lowbits 7--0`
> > +      - G\ :sub:`01lowbits 3--0`\ (bits 7--4) B\ :sub:`00highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`01highbits 7--0`
> > +      - B\ :sub:`02lowbits 7--0`
> > +      - G\ :sub:`03lowbits 3--0`\ (bits 7--4) B\ :sub:`02highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`03highbits 7--0`
> > +    * - start + 6:
> > +      - G\ :sub:`10lowbits 7--0`
> > +      - R\ :sub:`11lowbits 3--0`\ (bits 7--4) G\ :sub:`10highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`11highbits 7--0`
> > +      - G\ :sub:`12lowbits 7--0`
> > +      - R\ :sub:`13lowbits 3--0`\ (bits 7--4) G\ :sub:`12highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`13highbits 7--0`
> > +    * - start + 12:
> > +      - B\ :sub:`20lowbits 7--0`
> > +      - G\ :sub:`21lowbits 3--0`\ (bits 7--4) B\ :sub:`20highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`21highbits 7--0`
> > +      - B\ :sub:`22lowbits 7--0`
> > +      - G\ :sub:`23lowbits 3--0`\ (bits 7--4) B\ :sub:`22highbits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`23highbits 7--0`
> > +    * - start + 18:
> > +      - G\ :sub:`30lowbits 7--0`
> > +      - R\ :sub:`31lowbits 3--0`\ (bits 7--4) G\ :sub:`30highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`31highbits 7--0`
> > +      - G\ :sub:`32lowbits 7--0`
> > +      - R\ :sub:`33lowbits 3--0`\ (bits 7--4) G\ :sub:`32highbits 11--8`\ (bits 3--0)
> > +      - R\ :sub:`33highbits 7--0`
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > new file mode 100644
> > index 000000000000..c063de9f9ad8
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr12f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg12f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg12f:
> > +.. _v4l2-pix-fmt-mtisp-srggb12f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR12F ('MFBC'), V4L2_PIX_FMT_MTISP_SGBRG12F('MFGC'), V4L2_PIX_FMT_MTISP_SGRBG12F('MFgC'), V4L2_PIX_FMT_MTISP_SRGGB12F('MFRC')
> > +*******************************
> > +
> > +12-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 12 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 3--0`\ (bits 7--4) B\ :sub:`00high bits 11--8`\ (bits 3--0)
> > +    * - start + 2:
> > +      - FG\ :sub:`01high bits 7--0`
> > +      - G\ :sub:`02low bits 7--0`
> > +    * - start + 4:
> > +      - B\ :sub:`03low bits 3--0`\ (bits 7--4) G\ :sub:`02high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`03high bits 7--0`
> > +    * - start + 6:
> > +      - FG\ :sub:`04low bits 7--0`
> > +      - G\ :sub:`05low bits 3--0`\ (bits 7--4) FG\ :sub:`04high bits 11--8`\ (bits 3--0)
> > +    * - start + 8:
> > +      - G\ :sub:`05high bits 7--0`
> > +      -
> > +    * - start + 10:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 3--0`\ (bits 7--4) G\ :sub:`10high bits 11--8`\ (bits 3--0)
> > +    * - start + 12:
> > +      - R\ :sub:`11high bits 7--0`
> > +      - FG\ :sub:`12low bits 7--0`
> > +    * - start + 14:
> > +      - G\ :sub:`13low bits 3--0`\ (bits 7--4) FG\ :sub:`12high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`13high bits 7--0`
> > +    * - start + 16:
> > +      - R\ :sub:`14low bits 7--0`
> > +      - FG\ :sub:`15low bits 3--0`\ (bits 7--4) R\ :sub:`14high bits 11--8`\ (bits 3--0)
> > +    * - start + 18:
> > +      - FG\ :sub:`15high bits 7--0`
> > +      -
> > +    * - start + 20:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 3--0`\ (bits 7--4) B\ :sub:`20high bits 11--8`\ (bits 3--0)
> > +    * - start + 22:
> > +      - FG\ :sub:`21high bits 7--0`
> > +      - G\ :sub:`22low bits 7--0`
> > +    * - start + 24:
> > +      - B\ :sub:`23low bits 3--0`\ (bits 7--4) G\ :sub:`22high bits 11--8`\ (bits 3--0)
> > +      - B\ :sub:`23high bits 7--0`
> > +    * - start + 26:
> > +      - FG\ :sub:`24low bits 7--0`
> > +      - G\ :sub:`25low bits 3--0`\ (bits 7--4) FG\ :sub:`24high bits 11--8`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`25high bits 7--0`
> > +      -
> > +    * - start + 30:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 3--0`\ (bits 7--4) G\ :sub:`30high bits 11--8`\ (bits 3--0)
> > +    * - start + 32:
> > +      - R\ :sub:`31high bits 7--0`
> > +      - FG\ :sub:`32low bits 7--0`
> > +    * - start + 34:
> > +      - G\ :sub:`33low bits 3--0`\ (bits 7--4) FG\ :sub:`32high bits 11--8`\ (bits 3--0)
> > +      - G\ :sub:`33high bits 7--0`
> > +    * - start + 36:
> > +      - R\ :sub:`34low bits 7--0`
> > +      - FG\ :sub:`35low bits 3--0`\ (bits 7--4) R\ :sub:`34high bits 11--8`\ (bits 3--0)
> > +    * - start + 38:
> > +      - FG\ :sub:`35high bits 7--0`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > new file mode 100644
> > index 000000000000..39ea9882a792
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> > @@ -0,0 +1,73 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14:
> > +.. _v4l2-pix-fmt-mtisp-srggb14:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14 ('MBBE'), V4L2_PIX_FMT_MTISP_SGBRG14('MBGE'), V4L2_PIX_FMT_MTISP_SGRBG14('MBgE'), V4L2_PIX_FMT_MTISP_SRGGB14('MBRE')
> > +*******************************
> > +
> > +14-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 14 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +pixels cross the byte boundary and have a ratio of 7 bytes for each 4 pixels.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - G\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`01low bits 9--2`\
> > +      - B\ :sub:`02low bits 3--0`\ (bits 7--4) G\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - B\ :sub:`02low bits 11--4`\
> > +      - G\ :sub:`03low bits 5--0`\ (bits 7--2) B\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`03high bits 13--6`\
> > +      -
> > +    * - start + 8:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`\
> > +      - G\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 12:
> > +      - G\ :sub:`12low bits 11--4`\
> > +      - R\ :sub:`13low bits 5--0`\ (bits 7--2) G\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`13high bits 13--6`\
> > +      -
> > +    * - start + 16:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - G\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`21low bits 9--2`\
> > +      - B\ :sub:`22low bits 3--0`\ (bits 7--4) G\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 20:
> > +      - B\ :sub:`22low bits 11--4`\
> > +      - G\ :sub:`23low bits 5--0`\ (bits 7--2) B\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`23high bits 13--6`\
> > +      -
> > +    * - start + 24:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`\
> > +      - G\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`32low bits 11--4`\
> > +      - R\ :sub:`33low bits 5--0`\ (bits 7--2) G\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - R\ :sub:`33high bits 13--6`\
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > new file mode 100644
> > index 000000000000..010b1c190c60
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> > @@ -0,0 +1,110 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr14f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg14f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg14f:
> > +.. _v4l2-pix-fmt-mtisp-srggb14f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR14F ('MFBE'), V4L2_PIX_FMT_MTISP_SGBRG14F('MFGE'), V4L2_PIX_FMT_MTISP_SGRBG14F('MFgE'), V4L2_PIX_FMT_MTISP_SRGGB14F('MFRE')
> > +*******************************
> > +
> > +14-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 14 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00low bits 7--0`
> > +      - FG\ :sub:`01low bits 1--0`\ (bits 7--6) B\ :sub:`00high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`01low bits 9--2`
> > +      - G\ :sub:`02low bits 3--0`\ (bits 7--4) FG\ :sub:`01high bits 13--10`\ (bits 3--0)
> > +    * - start + 4:
> > +      - G\ :sub:`02low bits 11--4`
> > +      - B\ :sub:`03low bits 5--0`\ (bits 7--2) G\ :sub:`02high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`03high bits 13--6`
> > +      - FG\ :sub:`04low bits 7--0`
> > +    * - start + 8:
> > +      - G\ :sub:`05low bits 1--0`\ (bits 7--6) FG\ :sub:`04high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`05high bits 9--2`
> > +      - G\ :sub:`05high bits 13--10`
> > +      -
> > +    * - start + 12:
> > +      - G\ :sub:`10low bits 7--0`
> > +      - R\ :sub:`11low bits 1--0`\ (bits 7--6) G\ :sub:`10high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`11low bits 9--2`
> > +      - FG\ :sub:`12low bits 3--0`\ (bits 7--4) R\ :sub:`11high bits 13--10`\ (bits 3--0)
> > +    * - start + 16:
> > +      - FG\ :sub:`12low bits 11--4`
> > +      - G\ :sub:`13low bits 5--0`\ (bits 7--2) FG\ :sub:`12high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`13high bits 13--6`
> > +      - R\ :sub:`14low bits 7--0`
> > +    * - start + 20:
> > +      - FG\ :sub:`15low bits 1--0`\ (bits 7--6) R\ :sub:`14high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`15high bits 9--2`
> > +      - FG\ :sub:`15high bits 13--10`
> > +      -
> > +    * - start + 24:
> > +      - B\ :sub:`20low bits 7--0`
> > +      - FG\ :sub:`21low bits 1--0`\ (bits 7--6) B\ :sub:`20high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`21low bits 9--2`
> > +      - G\ :sub:`22low bits 3--0`\ (bits 7--4) FG\ :sub:`21high bits 13--10`\ (bits 3--0)
> > +    * - start + 28:
> > +      - G\ :sub:`22low bits 11--4`
> > +      - B\ :sub:`23low bits 5--0`\ (bits 7--2) G\ :sub:`22high bits 13--12`\ (bits 1--0)
> > +      - B\ :sub:`23high bits 13--6`
> > +      - FG\ :sub:`24low bits 7--0`
> > +    * - start + 32:
> > +      - G\ :sub:`25low bits 1--0`\ (bits 7--6) FG\ :sub:`24high bits 13--8`\ (bits 5--0)
> > +      - G\ :sub:`25high bits 9--2`
> > +      - G\ :sub:`25high bits 13--10`
> > +      -
> > +    * - start + 36:
> > +      - G\ :sub:`30low bits 7--0`
> > +      - R\ :sub:`31low bits 1--0`\ (bits 7--6) G\ :sub:`30high bits 13--8`\ (bits 5--0)
> > +      - R\ :sub:`31low bits 9--2`
> > +      - FG\ :sub:`32low bits 3--0`\ (bits 7--4) R\ :sub:`31high bits 13--10`\ (bits 3--0)
> > +    * - start + 40:
> > +      - FG\ :sub:`32low bits 11--4`
> > +      - G\ :sub:`33low bits 5--0`\ (bits 7--2) FG\ :sub:`32high bits 13--12`\ (bits 1--0)
> > +      - G\ :sub:`33high bits 13--6`
> > +      - R\ :sub:`34low bits 7--0`
> > +    * - start + 44:
> > +      - FG\ :sub:`35low bits 1--0`\ (bits 7--6) R\ :sub:`34high bits 13--8`\ (bits 5--0)
> > +      - FG\ :sub:`35high bits 9--2`
> > +      - FG\ :sub:`35high bits 13--10`
> > +      -
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > new file mode 100644
> > index 000000000000..86cadbf38175
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> > @@ -0,0 +1,51 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8:
> > +.. _v4l2-pix-fmt-mtisp-srggb8:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8 ('MBB8'), V4L2_PIX_FMT_MTISP_SGBRG8('MBG8'), V4L2_PIX_FMT_MTISP_SGRBG8('MBg8'), V4L2_PIX_FMT_MTISP_SRGGB8('MBR8')
> > +*******************************
> > +
> > +8-bit Packed Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format, meaning all the data bits for a pixel lying
> > +next to each other with no padding in memory, with a depth of 8 bits per pixel.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor.
> > +They are conventionally described as GRGR... BGBG..., RGRG... GBGB..., etc.
> > +Below is an example of conventional RGB byte order BGGR.
> 
> How do these 8-bit formats differ from the V4L2_PIX_FMT_SGBRG8 (and
> other variants) ? They seem identical based on the description.
> 

You are right. They are identical.
We will move "8-bit packed bayer formats" in next patch.

Best regards,

Jungo

> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - G\ :sub:`01`
> > +      - B\ :sub:`02`
> > +      - G\ :sub:`03`
> > +    * - start + 4:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - G\ :sub:`12`
> > +      - R\ :sub:`13`
> > +    * - start + 8:
> > +      - B\ :sub:`20`
> > +      - G\ :sub:`21`
> > +      - B\ :sub:`22`
> > +      - G\ :sub:`23`
> > +    * - start + 12:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - G\ :sub:`32`
> > +      - R\ :sub:`33`
> > \ No newline at end of file
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > new file mode 100644
> > index 000000000000..ca5151312bca
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> > @@ -0,0 +1,78 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-pix-fmt-mtisp-sbggr8f:
> > +.. _v4l2-pix-fmt-mtisp-sgbrg8f:
> > +.. _v4l2-pix-fmt-mtisp-sgrbg8f:
> > +.. _v4l2-pix-fmt-mtisp-srggb8f:
> > +
> > +*******************************
> > +V4L2_PIX_FMT_MTISP_SBGGR8F ('MFB8'), V4L2_PIX_FMT_MTISP_SGBRG8F('MFG8'), V4L2_PIX_FMT_MTISP_SGRBG8F('MFg8'), V4L2_PIX_FMT_MTISP_SRGGB8F('MFR8')
> > +*******************************
> > +
> > +8-bit Packed Full-G Bayer formats.
> > +
> > +Description
> > +===========
> > +
> > +These four pixel formats are used by Mediatek ISP P1.
> > +This is a packed format with a depth of 8 bits per sample with every 4 pixels.
> > +Full-G means 1 more pixel for green channel every 2 pixels.
> > +The least significant byte is stored at lower memory addresses (little-endian).
> > +The RGB byte order follows raw sRGB / Bayer format from sensor. They are conventionally
> > +described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an example of conventional
> > +RGB byte order BGGR.
> > +
> > +**Bit-packed representation.**
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +
> > +**Byte Order.**
> > +Each cell is one byte.
> > +
> > +.. flat-table::
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +
> > +    * - start + 0:
> > +      - B\ :sub:`00`
> > +      - FG\ :sub:`01`
> > +      - G\ :sub:`02`
> > +      - B\ :sub:`03`
> > +      - FG\ :sub:`04`
> > +      - G\ :sub:`05`
> > +    * - start + 6:
> > +      - G\ :sub:`10`
> > +      - R\ :sub:`11`
> > +      - FG\ :sub:`12`
> > +      - G\ :sub:`13`
> > +      - R\ :sub:`14`
> > +      - FG\ :sub:`15`
> > +    * - start + 12:
> > +      - B\ :sub:`20`
> > +      - FG\ :sub:`21`
> > +      - G\ :sub:`22`
> > +      - B\ :sub:`23`
> > +      - FG\ :sub:`24`
> > +      - G\ :sub:`25`
> > +    * - start + 18:
> > +      - G\ :sub:`30`
> > +      - R\ :sub:`31`
> > +      - FG\ :sub:`32`
> > +      - G\ :sub:`33`
> > +      - R\ :sub:`34`
> > +      - FG\ :sub:`35`
> > \ No newline at end of file
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
  2020-03-31 15:34       ` Helen Koike
  (?)
@ 2020-04-10 10:04         ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:04 UTC (permalink / raw)
  To: Helen Koike
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi, Helen:

Thanks for your comment.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hi Jungo,
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Add port node description in the dt-binding document.
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
> 
> It would be really nice to convert this to yaml.
> 
> For reference: https://lwn.net/Articles/771621/
> 
> Regards,
> Helen
> 

We will plan to covert txt to yaml.
Hopefully, we could overcome the learning cue of yaml.

Thanks,

Jungo

> >  1 file changed, 83 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..a85f37c0b87d
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,83 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp: The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- port: child port node corresponding to the data input, in accordance with
> > +  the video interface bindings defined in
> > +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> > +  node must contain at least one endpoint.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +				<0 0x1a003000 0 0x1000>,
> > +				<0 0x1a004000 0 0x2000>,
> > +				<0 0x1a006000 0 0x2000>,
> > +				<0 0x1a008000 0 0x2000>;
> > +		reg-names = "cam_sys",
> > +				"cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +				<&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "camsys_cam_cgpdn",
> > +				"camsys_camtg_cgpdn";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		mediatek,scp = <&scp>;
> > +
> > +		port {
> > +			camisp_endpoint: endpoint {
> > +				remote-endpoint = <&seninf_camisp_endpoint>;
> > +			};
> > +		};
> > +	};
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2020-04-10 10:04         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:04 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, shik,
	tfiga, zwisler, ddavenport

Hi, Helen:

Thanks for your comment.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hi Jungo,
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Add port node description in the dt-binding document.
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
> 
> It would be really nice to convert this to yaml.
> 
> For reference: https://lwn.net/Articles/771621/
> 
> Regards,
> Helen
> 

We will plan to covert txt to yaml.
Hopefully, we could overcome the learning cue of yaml.

Thanks,

Jungo

> >  1 file changed, 83 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..a85f37c0b87d
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,83 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp: The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- port: child port node corresponding to the data input, in accordance with
> > +  the video interface bindings defined in
> > +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> > +  node must contain at least one endpoint.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +				<0 0x1a003000 0 0x1000>,
> > +				<0 0x1a004000 0 0x2000>,
> > +				<0 0x1a006000 0 0x2000>,
> > +				<0 0x1a008000 0 0x2000>;
> > +		reg-names = "cam_sys",
> > +				"cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +				<&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "camsys_cam_cgpdn",
> > +				"camsys_camtg_cgpdn";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		mediatek,scp = <&scp>;
> > +
> > +		port {
> > +			camisp_endpoint: endpoint {
> > +				remote-endpoint = <&seninf_camisp_endpoint>;
> > +			};
> > +		};
> > +	};
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1
@ 2020-04-10 10:04         ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:04 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, shik,
	tfiga, zwisler, ddavenport

Hi, Helen:

Thanks for your comment.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hi Jungo,
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > This patch adds DT binding document for the Pass 1 (P1) unit
> > in Mediatek's camera ISP system. The Pass 1 unit grabs the sensor
> > data out from the sensor interface, applies ISP image effects
> > from tuning data and outputs the image data or statistics data to DRAM.
> > 
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> > ---
> > Changes from v6:
> >  - Add port node description in the dt-binding document.
> > ---
> >  .../bindings/media/mediatek,camisp.txt        | 83 +++++++++++++++++++
> 
> It would be really nice to convert this to yaml.
> 
> For reference: https://lwn.net/Articles/771621/
> 
> Regards,
> Helen
> 

We will plan to covert txt to yaml.
Hopefully, we could overcome the learning cue of yaml.

Thanks,

Jungo

> >  1 file changed, 83 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/mediatek,camisp.txt b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > new file mode 100644
> > index 000000000000..a85f37c0b87d
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/mediatek,camisp.txt
> > @@ -0,0 +1,83 @@
> > +* Mediatek Image Signal Processor Pass 1 (ISP P1)
> > +
> > +The Pass 1 unit of Mediatek's camera ISP system grabs the sensor data out
> > +from the sensor interface, applies ISP effects from tuning data and outputs
> > +the image data and statistics data to DRAM. Furthermore, Pass 1 unit has
> > +the ability to output two different resolutions frames at the same time to
> > +increase the performance of the camera application.
> > +
> > +Required properties:
> > +- compatible: Must be "mediatek,mt8183-camisp" for MT8183.
> > +- reg: Physical base address of the camera function block register and
> > +  length of memory mapped region. Must contain an entry for each entry
> > +  in reg-names.
> > +- reg-names: Must include the following entries:
> > +  "cam_sys": Camera base function block
> > +  "cam_uni": Camera UNI function block
> > +  "cam_a": Camera ISP P1 hardware unit A
> > +  "cam_b": Camera ISP P1 hardware unit B
> > +  "cam_c": Camera ISP P1 hardware unit C
> > +- interrupts: Must contain an entry for each entry in interrupt-names.
> > +- interrupt-names : Must include the following entries:
> > +  "cam_uni": Camera UNI interrupt
> > +  "cam_a": Camera unit A interrupt
> > +  "cam_b": Camera unit B interrupt
> > +  "cam_c": Camera unit C interrupt
> > +- iommus: Shall point to the respective IOMMU block with master port
> > +  as argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > +  for details.
> > +- clocks: A list of phandle and clock specifier pairs as listed
> > +  in clock-names property, see
> > +  Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
> > +- clock-names: Must be "camsys_cam_cgpdn" and "camsys_camtg_cgpdn".
> > +- mediatek,larb: Must contain the local arbiters in the current SoCs, see
> > +  Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.txt
> > +  for details.
> > +- power-domains: a phandle to the power domain, see
> > +  Documentation/devicetree/bindings/power/power_domain.txt for details.
> > +- mediatek,scp: The node of system control processor (SCP), see
> > +  Documentation/devicetree/bindings/remoteproc/mtk,scp.txt for details.
> > +- port: child port node corresponding to the data input, in accordance with
> > +  the video interface bindings defined in
> > +  Documentation/devicetree/bindings/media/video-interfaces.txt. The port
> > +  node must contain at least one endpoint.
> > +
> > +Example:
> > +SoC specific DT entry:
> > +
> > +	camisp: camisp@1a000000 {
> > +		compatible = "mediatek,mt8183-camisp";
> > +		reg = <0 0x1a000000 0 0x1000>,
> > +				<0 0x1a003000 0 0x1000>,
> > +				<0 0x1a004000 0 0x2000>,
> > +				<0 0x1a006000 0 0x2000>,
> > +				<0 0x1a008000 0 0x2000>;
> > +		reg-names = "cam_sys",
> > +				"cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		interrupts = <GIC_SPI 253 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 254 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 255 IRQ_TYPE_LEVEL_LOW>,
> > +				<GIC_SPI 256 IRQ_TYPE_LEVEL_LOW>;
> > +		interrupt-names = "cam_uni",
> > +				"cam_a",
> > +				"cam_b",
> > +				"cam_c";
> > +		iommus = <&iommu M4U_PORT_CAM_IMGO>;
> > +		clocks = <&camsys CLK_CAM_CAM>,
> > +				<&camsys CLK_CAM_CAMTG>;
> > +		clock-names = "camsys_cam_cgpdn",
> > +				"camsys_camtg_cgpdn";
> > +		mediatek,larb = <&larb3>,
> > +				<&larb6>;
> > +		power-domains = <&scpsys MT8183_POWER_DOMAIN_CAM>;
> > +		mediatek,scp = <&scp>;
> > +
> > +		port {
> > +			camisp_endpoint: endpoint {
> > +				remote-endpoint = <&seninf_camisp_endpoint>;
> > +			};
> > +		};
> > +	};
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
  2020-03-31 15:34     ` Helen Koike
@ 2020-04-10 10:32       ` Jungo Lin
  -1 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:32 UTC (permalink / raw)
  To: Helen Koike
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Helen:

Thanks for your comment.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hi Jungo,
> 
> I was taking a look at this patchset, please see my comments below.
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > Hello,
> > 
> > This patch series adding the driver for Pass 1 (P1) unit in
> > Mediatek's camera ISP system on mt8183 SoC, which will be used in
> > camera features of CrOS.
> > 
> > Pass 1 unit processes image signal from sensor devices and accepts the
> > tuning parameters to adjust the image quality. It performs optical
> > black correction, defect pixel correction, W/IR imbalance correction
> > and lens shading correction for RAW processing.
> > 
> > The driver is implemented with V4L2 and media controller framework so
> > we have the following entities to describe the ISP pass 1 path.
> > 
> > (The current metadata interface used in meta input and partial meta
> > nodes is only a temporary solution to kick off the driver development
> > and is not ready to be reviewed yet.)
> > 
> > 1. meta input (output video device): connect to ISP P1 sub device.
> > It accepts the tuning buffer from user.
> > 
> > 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> > main stream and packed out video devices. When processing an image,
> > Pass 1 hardware supports multiple output images with different sizes
> > and formats so it needs two capture video devices ("main stream" and
> > "packed out") to return the image data to the user.
> > 
> > 3. main stream (capture video device): return the processed image data
> > which is used in capture scenario.
> > 
> > 4. packed out (capture video device): return the processed image data
> > which is used in preview scenario.
> > 
> > 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> > 
> > 6. partial meta 1 (capture video device): return the AF statistics.
> > 
> > 7. partial meta 2 (capture video device): return the local contrast
> >    enhanced statistics.
> > 
> > 8. partial meta 3 (capture video device): return the local motion
> >    vector statistics.
> > 
> > The overall patches of the series is:
> > 
> > * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> > * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> > * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> > * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >   Moreover, implement standard V4L2 video driver that utilizes
> >   V4L2 and media framework APIs. Communicate with co-process via SCP
> >   communication to compose ISP registers in the firmware.
> > 
> > Here is ISP P1 media topology:
> > It is included the main/sub sensor, sen-inf sub-devices and len device
> > which are implemented in below patch[1][2][3][4]:
> 
> I would be nice if you could provide a branch with those applied.
> 

We apply those patches in the chromeos-4.19 to test.
https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19


> > 
> > For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> > patch sets.
> > 
> > /usr/bin/media-ctl -p -d /dev/media2
> > 
> > Media controller API version 4.19.89
> > 
> > Media device information
> > ------------------------
> > driver          mtk-cam-p1
> > model           mtk-cam-p1
> > serial          
> > bus info        platform:1a000000.camisp
> > hw revision     0x0
> > driver version  4.19.89
> > 
> > Device topology
> > - entity 1: mtk-cam-p1 (12 pads, 8 links)
> 
> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> Is this correct?
> 
> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> 

Currently, we only support single-cam in this SoC with upstream driver.
It is plan in next Mediatek SoC to support multi-cam capabilities. So
we'd like to keep the naming to avoid confusion.

> >             type V4L2 subdev subtype Unknown flags 0
> >             device node name /dev/v4l-subdev0
> > 	pad0: Sink
> > 		<- "mtk-cam-p1 meta input":0 []
> 
> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> 

Ok, we will revise our naming in next patch.

> > 	pad1: Source
> > 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> 
> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> 

Yes, you are right. We will remove IMMUTABLE flag in next patch.

> > 	pad2: Source
> > 		-> "mtk-cam-p1 packed out":0 []
> 
> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> 

Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
padding in the memory. We may revise the naming in next patch.

> > 	pad3: Source
> > 		-> "mtk-cam-p1 partial meta 0":0 []
> > 	pad4: Source
> > 		-> "mtk-cam-p1 partial meta 1":0 []
> > 	pad5: Source
> > 		-> "mtk-cam-p1 partial meta 2":0 []
> > 	pad6: Source
> > 		-> "mtk-cam-p1 partial meta 3":0 []
> 
> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> 
> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> 
> I also would prefer to remove blank spaces.
> 
> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> 

No, these links are optional to setup for userspace.
For naming part, we will align with driver source codes.

> > 	pad7: Source
> > 	pad8: Source
> > 	pad9: Source
> > 	pad10: Source
> 
> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> 

These pads are just reserved purpose.
We will plan to remove them in next patch.

Thanks,

Jungo

> Regards,
> Helen
> 
> > 	pad11: Sink
> > 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> > 
> > - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video2
> > 	pad0: Source
> > 		-> "mtk-cam-p1":0 []
> > 
> > - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video3
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> > 
> > - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video4
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":2 []
> > 
> > - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video5
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":3 []
> > 
> > - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video6
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":4 []
> > 
> > - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video7
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":5 []
> > 
> > - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video8
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":6 []
> > 
> > - entity 56: 1a040000.seninf (12 pads, 3 links)
> >              type V4L2 subdev subtype Unknown flags 0
> >              device node name /dev/v4l-subdev1
> > 	pad0: Sink
> > 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> > 		<- "ov8856 2-0010":0 [ENABLED]
> > 	pad1: Sink
> > 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> > 		<- "ov02a10 4-003d":0 []
> > 	pad2: Sink
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad3: Sink
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad4: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> > 	pad5: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad6: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad7: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad8: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad9: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad10: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad11: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 
> > - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev2
> > 	pad0: Source
> > 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> > 		-> "1a040000.seninf":0 [ENABLED]
> > 
> > - entity 73: dw9768 2-000c (0 pad, 0 link)
> >              type V4L2 subdev subtype Lens flags 0
> >              device node name /dev/v4l-subdev3
> > 
> > - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev4
> > 	pad0: Source
> > 		[fmt:SRGGB10_1X10/1600x1200 field:none]
> > 		-> "1a040000.seninf":1 []
> > 
> > ===========
> > = history =
> > ===========
> > 
> > version 6:
> >  - Add port node description in the dt-binding document and device tree
> >    by Tomasz Figa.
> >  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >    version.
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >  - Correct auto suspend timer value for suspend/resume issue.
> >  - Increase IPI guard timer to 1 second to avoid false alarm command
> >    timeout event.
> >  - Fix KE due to no sen-inf sub-device.
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation.
> >  - Add rst documents for Mediatek meta formats.
> >  - New meta buffer structure design & re-factoring.
> > 
> > version 5:
> >  - Fixed Rob's comment on dt-binding format
> >  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >    and new timestamp type in driver
> >  - Fix buffer en-queue timing issue in v4
> >  - Remove default link_notify callback function in mtk_cam_media_ops
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> >  - Align and pack IPI command structures for EC ROM size shrink
> > 
> > version 4:
> >  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >    patch[4]
> >  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >  - Extend Mediatek proprietary image formats to support bayer order
> >  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> >  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >  - Align and pack IPI command structures for EC ROM size shrink
> > 
> > version 3:
> >  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >    reserved memory region. (Rob Herring)
> >  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >  - Revise ISP Pass1 Kconfig
> >  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >  - Fix kernel warning messages when running v4l2_compliance test
> >  - Move AFO buffer enqueue & de-queue from request API to non-request
> >  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >    declaration (Hans Verkuil)
> >    Split GET_BIN_INFO control into two controls to get width & height
> >    in-dependently (Hans Verkuil)
> >  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >    Fix Drew's comments which are addressed in v2 patch
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >    Fix Drew's comments which are addressed in v2 patch
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >  - mtk_cam-scp.h / mtk_cam-scp.c
> >    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >    Fix ISP de-initialize timing KE issue
> >  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >    Get the reserved shared memory via SCP driver (Tomasz Figa)
> > 
> > Todo:
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> > 
> > version 2:
> >  - Add 3A enhancement feature which includes:
> >    Separates 3A pipeline out of frame basis to improve
> >    AE/AWB (exposure and white balance) performance.
> >    Add 2 SCP sub-commands for 3A meta buffers.
> >  - Add new child device to manage P1 shared memory between P1 HW unit
> >    and co-processor.
> >  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >  - Revised document wording for dt-bindings documents & dts information.
> >  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >  - mtk_cam-dev.h / mtk_cam-dev.c
> >    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >    or add comments.
> >    Revised buffer size for LMVO & LCSO.
> >    Fix pixel format utility function.
> >    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >  - mtk_cam-v4l2-util.c
> >    Refactoring V4L2 async mechanism with seninf driver only
> >    Refactoring CIO (Connection IO) implementation with active sensor
> >    Revised stream on function for 3A enhancement feature
> >    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >    Add meta buffer index register definitions
> >    Add meta DMA configuration function.
> >    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >    Add isp_setup_scp_rproc function to get RPC handle
> >    Add mtk_cam_reserved_memory_init for shared memory management
> >  - mtk_cam-scp.h / mtk_cam-scp.c
> >    Add new meta strictures for 3A enhancement feature
> >    Add new IPI command utility function for 3A enhancement feature
> >    Enhance isp_composer_dma_sg_init function flow
> >    Shorten overall IPI command structure size
> >    Remove scp_state state checking
> >    Improve code readability
> >  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >    memory region.
> > 
> > TODOs:
> >  - 3A enhancement feature bug fixing
> > 
> > version 1:
> >  - Revised driver sources based on Tomasz's comments including
> >    part1/2/3/4 in RFC V0 patch.
> >  - Remove DMA cache mechanism.
> >    Support two new video devices (LCSO/LMVO) for advance camera
> >    features.
> >  - Fixed v4l2-compliance test failure items.
> >  - Add private controls for Mediatek camera middle-ware.
> >  - Replace VPU driver's APIs with new SCP driver interface for
> >    co-processor communication.
> >  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >    commands RX handling.
> >  - Fix internal bugs.
> > 
> > TODOs:
> >  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >    for shared memory management.
> >  - Revised file names.
> >  - Support non frame-sync AFO/AAO DMA buffers
> > 
> > version 0:
> > - Initial submission
> > 
> > ==================
> >  Dependent patch set
> > ==================
> > 
> > Camera ISP P1 driver depends on seninf driver, SCP driver.
> > The patches are listed as following:
> > 
> > [1]. media: support Mediatek sensor interface driver
> > https://patchwork.kernel.org/cover/11145845/
> > 
> > [2]. media: ov8856: Add YAML binding and sensor mode support
> > https://patchwork.kernel.org/cover/11220785/
> > 
> > [3]. media: i2c: Add support for OV02A10 sensor
> > https://patchwork.kernel.org/cover/11284779/
> > 
> > [4]. media: i2c: add support for DW9768 VCM driver
> > https://patchwork.kernel.org/cover/11132299/
> > 
> > [5]. Add support for mt8183 SCP
> > https://patchwork.kernel.org/cover/11239065/
> > 
> > [6]. MT8183 IOMMU SUPPORT
> > https://patchwork.kernel.org/cover/11112765/
> > 
> > ==================
> >  Compliance test
> > ==================
> > 
> > The v4l2-compliance is built with the below lastest patch.
> > https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> > 
> > Note 1.
> > This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> > 
> > Note 2.
> > For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> > called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> > 
> > Note 3.
> > The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> > 
> > /usr/bin/v4l2-compliance -m /dev/media2
> > 
> > v4l2-compliance SHA: not available, 32 bits
> > 
> > Compliance test for mtk-cam-p1 device /dev/media1:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           :
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.67
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.67
> > 
> > Required ioctls:
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/media1 open: OK
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 	test for unlimited opens: OK
> > 
> > Media Controller ioctls:
> > 	test MEDIA_IOC_G_TOPOLOGY: OK
> > 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> > 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> > 	test MEDIA_IOC_SETUP_LINK: OK
> > 
> > Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video25:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.67
> > 	Capabilities     : 0x8c200000
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x0c200000
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.67
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.67
> > Interface Info:
> > 	ID               : 0x03000010
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000000e (14)
> > 	Name             : mtk-cam-p1 meta input
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100000f   : 0: Source
> > 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video25 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composiv4l2-compliance SHA: not available, 32 bits
> > 
> > Compliance test for mtk-cam-p1 device /dev/media2:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > 
> > Required ioctls:
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/media2 open: OK
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 	test for unlimited opens: OK
> > 
> > Media Controller ioctls:
> > 	test MEDIA_IOC_G_TOPOLOGY: OK
> > 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> > 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> > 	test MEDIA_IOC_SETUP_LINK: OK
> > 
> > Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video2:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x8c200000
> > 		Metadata Output
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x0c200000
> > 		Metadata Output
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000010
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000000e (14)
> > 	Name             : mtk-cam-p1 meta input
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100000f   : 0: Source
> > 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video2 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video3:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000016
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000014 (20)
> > 	Name             : mtk-cam-p1 main stream
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000015   : 0: Sink
> > 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video3 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video4:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x0300001c
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000001a (26)
> > 	Name             : mtk-cam-p1 packed out
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100001b   : 0: Sink
> > 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video4 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video5:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000022
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000020 (32)
> > 	Name             : mtk-cam-p1 partial meta 0
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000021   : 0: Sink
> > 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video5 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video6:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000028
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000026 (38)
> > 	Name             : mtk-cam-p1 partial meta 1
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000027   : 0: Sink
> > 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video6 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video7:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x0300002e
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000002c (44)
> > 	Name             : mtk-cam-p1 partial meta 2
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100002d   : 0: Sink
> > 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video7 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video8:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000034
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000032 (50)
> > 	Name             : mtk-cam-p1 partial meta 3
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000033   : 0: Sink
> > 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video8 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000050
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000001 (1)
> > 	Name             : mtk-cam-p1
> > 	Function         : Video Pixel Formatter
> > 	Pad 0x01000002   : 0: Sink
> > 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> > 	Pad 0x01000003   : 1: Source
> > 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> > 	Pad 0x01000004   : 2: Source
> > 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> > 	Pad 0x01000005   : 3: Source
> > 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> > 	Pad 0x01000006   : 4: Source
> > 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> > 	Pad 0x01000007   : 5: Source
> > 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> > 	Pad 0x01000008   : 6: Source
> > 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> > 	Pad 0x01000009   : 7: Source
> > 	Pad 0x0100000a   : 8: Source
> > 	Pad 0x0100000b   : 9: Source
> > 	Pad 0x0100000c   : 10: Source
> > 	Pad 0x0100000d   : 11: Sink
> > 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev0 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 1):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 2):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 3):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 4):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 5):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 6):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 7):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 8):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 9):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 10):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 11):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000052
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000038 (56)
> > 	Name             : 1a040000.seninf
> > 	Function         : Video Interface Bridge
> > 	Pad 0x01000039   : 0: Sink
> > 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> > 	Pad 0x0100003a   : 1: Sink
> > 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> > 	Pad 0x0100003b   : 2: Sink
> > 	Pad 0x0100003c   : 3: Sink
> > 	Pad 0x0100003d   : 4: Source
> > 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> > 	Pad 0x0100003e   : 5: Source
> > 	Pad 0x0100003f   : 6: Source
> > 	Pad 0x01000040   : 7: Source
> > 	Pad 0x01000041   : 8: Source
> > 	Pad 0x01000042   : 9: Source
> > 	Pad 0x01000043   : 10: Source
> > 	Pad 0x01000044   : 11: Source
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev1 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 1):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 2):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 3):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 4):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 5):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 6):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 7):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 8):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 9):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 10):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 11):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 2 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000054
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000045 (69)
> > 	Name             : ov8856 2-0010
> > 	Function         : Camera Sensor
> > 	Pad 0x01000046   : 0: Source
> > 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev2 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 0):
> > 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> > 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> > 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> > 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> > 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 11 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000056
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000049 (73)
> > 	Name             : dw9768 2-000c
> > 	Function         : Lens Controller
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev3 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 2 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000058
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x0000004a (74)
> > 	Name             : ov02a10 4-003d
> > 	Function         : Camera Sensor
> > 	Pad 0x0100004b   : 0: Source
> > 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev4 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 		fail: v4l2-test-controls.cpp(362): returned control value out of range
> > 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> > 	test VIDIOC_G/S_CTRL: FAIL
> > 		fail: v4l2-test-controls.cpp(549): returned control value out of range
> > 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 10 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> > 
> > Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> > 
> > 
> > Jungo Lin (5):
> >   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >   media: videodev2.h: Add new boottime timestamp type
> >   media: platform: Add Mediatek ISP P1 image & meta formats
> >   media: platform: Add Mediatek ISP P1 V4L2 device driver
> > 
> >  .../bindings/media/mediatek,camisp.txt        |   83 +
> >  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >  include/uapi/linux/videodev2.h                |   41 +
> >  24 files changed, 4226 insertions(+), 1 deletion(-)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-04-10 10:32       ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-04-10 10:32 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, shik,
	tfiga, zwisler, ddavenport

Hi Helen:

Thanks for your comment.

On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> Hi Jungo,
> 
> I was taking a look at this patchset, please see my comments below.
> 
> On 12/19/19 3:49 AM, Jungo Lin wrote:
> > Hello,
> > 
> > This patch series adding the driver for Pass 1 (P1) unit in
> > Mediatek's camera ISP system on mt8183 SoC, which will be used in
> > camera features of CrOS.
> > 
> > Pass 1 unit processes image signal from sensor devices and accepts the
> > tuning parameters to adjust the image quality. It performs optical
> > black correction, defect pixel correction, W/IR imbalance correction
> > and lens shading correction for RAW processing.
> > 
> > The driver is implemented with V4L2 and media controller framework so
> > we have the following entities to describe the ISP pass 1 path.
> > 
> > (The current metadata interface used in meta input and partial meta
> > nodes is only a temporary solution to kick off the driver development
> > and is not ready to be reviewed yet.)
> > 
> > 1. meta input (output video device): connect to ISP P1 sub device.
> > It accepts the tuning buffer from user.
> > 
> > 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> > main stream and packed out video devices. When processing an image,
> > Pass 1 hardware supports multiple output images with different sizes
> > and formats so it needs two capture video devices ("main stream" and
> > "packed out") to return the image data to the user.
> > 
> > 3. main stream (capture video device): return the processed image data
> > which is used in capture scenario.
> > 
> > 4. packed out (capture video device): return the processed image data
> > which is used in preview scenario.
> > 
> > 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> > 
> > 6. partial meta 1 (capture video device): return the AF statistics.
> > 
> > 7. partial meta 2 (capture video device): return the local contrast
> >    enhanced statistics.
> > 
> > 8. partial meta 3 (capture video device): return the local motion
> >    vector statistics.
> > 
> > The overall patches of the series is:
> > 
> > * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> > * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> > * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> > * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >   Moreover, implement standard V4L2 video driver that utilizes
> >   V4L2 and media framework APIs. Communicate with co-process via SCP
> >   communication to compose ISP registers in the firmware.
> > 
> > Here is ISP P1 media topology:
> > It is included the main/sub sensor, sen-inf sub-devices and len device
> > which are implemented in below patch[1][2][3][4]:
> 
> I would be nice if you could provide a branch with those applied.
> 

We apply those patches in the chromeos-4.19 to test.
https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19


> > 
> > For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> > patch sets.
> > 
> > /usr/bin/media-ctl -p -d /dev/media2
> > 
> > Media controller API version 4.19.89
> > 
> > Media device information
> > ------------------------
> > driver          mtk-cam-p1
> > model           mtk-cam-p1
> > serial          
> > bus info        platform:1a000000.camisp
> > hw revision     0x0
> > driver version  4.19.89
> > 
> > Device topology
> > - entity 1: mtk-cam-p1 (12 pads, 8 links)
> 
> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> Is this correct?
> 
> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> 

Currently, we only support single-cam in this SoC with upstream driver.
It is plan in next Mediatek SoC to support multi-cam capabilities. So
we'd like to keep the naming to avoid confusion.

> >             type V4L2 subdev subtype Unknown flags 0
> >             device node name /dev/v4l-subdev0
> > 	pad0: Sink
> > 		<- "mtk-cam-p1 meta input":0 []
> 
> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> 

Ok, we will revise our naming in next patch.

> > 	pad1: Source
> > 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> 
> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> 

Yes, you are right. We will remove IMMUTABLE flag in next patch.

> > 	pad2: Source
> > 		-> "mtk-cam-p1 packed out":0 []
> 
> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> 

Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
padding in the memory. We may revise the naming in next patch.

> > 	pad3: Source
> > 		-> "mtk-cam-p1 partial meta 0":0 []
> > 	pad4: Source
> > 		-> "mtk-cam-p1 partial meta 1":0 []
> > 	pad5: Source
> > 		-> "mtk-cam-p1 partial meta 2":0 []
> > 	pad6: Source
> > 		-> "mtk-cam-p1 partial meta 3":0 []
> 
> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> 
> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> 
> I also would prefer to remove blank spaces.
> 
> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> 

No, these links are optional to setup for userspace.
For naming part, we will align with driver source codes.

> > 	pad7: Source
> > 	pad8: Source
> > 	pad9: Source
> > 	pad10: Source
> 
> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> 

These pads are just reserved purpose.
We will plan to remove them in next patch.

Thanks,

Jungo

> Regards,
> Helen
> 
> > 	pad11: Sink
> > 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> > 
> > - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video2
> > 	pad0: Source
> > 		-> "mtk-cam-p1":0 []
> > 
> > - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video3
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> > 
> > - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video4
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":2 []
> > 
> > - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video5
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":3 []
> > 
> > - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video6
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":4 []
> > 
> > - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video7
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":5 []
> > 
> > - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video8
> > 	pad0: Sink
> > 		<- "mtk-cam-p1":6 []
> > 
> > - entity 56: 1a040000.seninf (12 pads, 3 links)
> >              type V4L2 subdev subtype Unknown flags 0
> >              device node name /dev/v4l-subdev1
> > 	pad0: Sink
> > 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> > 		<- "ov8856 2-0010":0 [ENABLED]
> > 	pad1: Sink
> > 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> > 		<- "ov02a10 4-003d":0 []
> > 	pad2: Sink
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad3: Sink
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad4: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> > 	pad5: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad6: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad7: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad8: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad9: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad10: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 	pad11: Source
> > 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> > 
> > - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev2
> > 	pad0: Source
> > 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> > 		-> "1a040000.seninf":0 [ENABLED]
> > 
> > - entity 73: dw9768 2-000c (0 pad, 0 link)
> >              type V4L2 subdev subtype Lens flags 0
> >              device node name /dev/v4l-subdev3
> > 
> > - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev4
> > 	pad0: Source
> > 		[fmt:SRGGB10_1X10/1600x1200 field:none]
> > 		-> "1a040000.seninf":1 []
> > 
> > ===========
> > = history =
> > ===========
> > 
> > version 6:
> >  - Add port node description in the dt-binding document and device tree
> >    by Tomasz Figa.
> >  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >    version.
> >  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >  - Correct auto suspend timer value for suspend/resume issue.
> >  - Increase IPI guard timer to 1 second to avoid false alarm command
> >    timeout event.
> >  - Fix KE due to no sen-inf sub-device.
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation.
> >  - Add rst documents for Mediatek meta formats.
> >  - New meta buffer structure design & re-factoring.
> > 
> > version 5:
> >  - Fixed Rob's comment on dt-binding format
> >  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >    and new timestamp type in driver
> >  - Fix buffer en-queue timing issue in v4
> >  - Remove default link_notify callback function in mtk_cam_media_ops
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> >  - Align and pack IPI command structures for EC ROM size shrink
> > 
> > version 4:
> >  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >    patch[4]
> >  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >  - Extend Mediatek proprietary image formats to support bayer order
> >  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> > 
> > Todo:
> >  - vb2_ops's buf_request_complete callback function implementation
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> >  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >  - Align and pack IPI command structures for EC ROM size shrink
> > 
> > version 3:
> >  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >    reserved memory region. (Rob Herring)
> >  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >  - Revise ISP Pass1 Kconfig
> >  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >  - Fix kernel warning messages when running v4l2_compliance test
> >  - Move AFO buffer enqueue & de-queue from request API to non-request
> >  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >    declaration (Hans Verkuil)
> >    Split GET_BIN_INFO control into two controls to get width & height
> >    in-dependently (Hans Verkuil)
> >  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >    Fix Drew's comments which are addressed in v2 patch
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >    Fix Drew's comments which are addressed in v2 patch
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >  - mtk_cam-scp.h / mtk_cam-scp.c
> >    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >    Fix ISP de-initialize timing KE issue
> >  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >    Get the reserved shared memory via SCP driver (Tomasz Figa)
> > 
> > Todo:
> >  - Add rst documents for Mediatek meta formats
> >  - New meta buffer structure design & re-factoring
> > 
> > version 2:
> >  - Add 3A enhancement feature which includes:
> >    Separates 3A pipeline out of frame basis to improve
> >    AE/AWB (exposure and white balance) performance.
> >    Add 2 SCP sub-commands for 3A meta buffers.
> >  - Add new child device to manage P1 shared memory between P1 HW unit
> >    and co-processor.
> >  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >  - Revised document wording for dt-bindings documents & dts information.
> >  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >  - mtk_cam-dev.h / mtk_cam-dev.c
> >    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >    or add comments.
> >    Revised buffer size for LMVO & LCSO.
> >    Fix pixel format utility function.
> >    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >  - mtk_cam-v4l2-util.c
> >    Refactoring V4L2 async mechanism with seninf driver only
> >    Refactoring CIO (Connection IO) implementation with active sensor
> >    Revised stream on function for 3A enhancement feature
> >    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >    Add meta buffer index register definitions
> >    Add meta DMA configuration function.
> >    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >    Add isp_setup_scp_rproc function to get RPC handle
> >    Add mtk_cam_reserved_memory_init for shared memory management
> >  - mtk_cam-scp.h / mtk_cam-scp.c
> >    Add new meta strictures for 3A enhancement feature
> >    Add new IPI command utility function for 3A enhancement feature
> >    Enhance isp_composer_dma_sg_init function flow
> >    Shorten overall IPI command structure size
> >    Remove scp_state state checking
> >    Improve code readability
> >  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >    memory region.
> > 
> > TODOs:
> >  - 3A enhancement feature bug fixing
> > 
> > version 1:
> >  - Revised driver sources based on Tomasz's comments including
> >    part1/2/3/4 in RFC V0 patch.
> >  - Remove DMA cache mechanism.
> >    Support two new video devices (LCSO/LMVO) for advance camera
> >    features.
> >  - Fixed v4l2-compliance test failure items.
> >  - Add private controls for Mediatek camera middle-ware.
> >  - Replace VPU driver's APIs with new SCP driver interface for
> >    co-processor communication.
> >  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >    commands RX handling.
> >  - Fix internal bugs.
> > 
> > TODOs:
> >  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >    for shared memory management.
> >  - Revised file names.
> >  - Support non frame-sync AFO/AAO DMA buffers
> > 
> > version 0:
> > - Initial submission
> > 
> > ==================
> >  Dependent patch set
> > ==================
> > 
> > Camera ISP P1 driver depends on seninf driver, SCP driver.
> > The patches are listed as following:
> > 
> > [1]. media: support Mediatek sensor interface driver
> > https://patchwork.kernel.org/cover/11145845/
> > 
> > [2]. media: ov8856: Add YAML binding and sensor mode support
> > https://patchwork.kernel.org/cover/11220785/
> > 
> > [3]. media: i2c: Add support for OV02A10 sensor
> > https://patchwork.kernel.org/cover/11284779/
> > 
> > [4]. media: i2c: add support for DW9768 VCM driver
> > https://patchwork.kernel.org/cover/11132299/
> > 
> > [5]. Add support for mt8183 SCP
> > https://patchwork.kernel.org/cover/11239065/
> > 
> > [6]. MT8183 IOMMU SUPPORT
> > https://patchwork.kernel.org/cover/11112765/
> > 
> > ==================
> >  Compliance test
> > ==================
> > 
> > The v4l2-compliance is built with the below lastest patch.
> > https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> > 
> > Note 1.
> > This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> > 
> > Note 2.
> > For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> > called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> > 
> > Note 3.
> > The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> > 
> > /usr/bin/v4l2-compliance -m /dev/media2
> > 
> > v4l2-compliance SHA: not available, 32 bits
> > 
> > Compliance test for mtk-cam-p1 device /dev/media1:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           :
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.67
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.67
> > 
> > Required ioctls:
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/media1 open: OK
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 	test for unlimited opens: OK
> > 
> > Media Controller ioctls:
> > 	test MEDIA_IOC_G_TOPOLOGY: OK
> > 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> > 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> > 	test MEDIA_IOC_SETUP_LINK: OK
> > 
> > Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video25:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.67
> > 	Capabilities     : 0x8c200000
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x0c200000
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.67
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.67
> > Interface Info:
> > 	ID               : 0x03000010
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000000e (14)
> > 	Name             : mtk-cam-p1 meta input
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100000f   : 0: Source
> > 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video25 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composiv4l2-compliance SHA: not available, 32 bits
> > 
> > Compliance test for mtk-cam-p1 device /dev/media2:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > 
> > Required ioctls:
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/media2 open: OK
> > 	test MEDIA_IOC_DEVICE_INFO: OK
> > 	test for unlimited opens: OK
> > 
> > Media Controller ioctls:
> > 	test MEDIA_IOC_G_TOPOLOGY: OK
> > 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> > 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> > 	test MEDIA_IOC_SETUP_LINK: OK
> > 
> > Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video2:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x8c200000
> > 		Metadata Output
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x0c200000
> > 		Metadata Output
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000010
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000000e (14)
> > 	Name             : mtk-cam-p1 meta input
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100000f   : 0: Source
> > 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video2 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video3:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000016
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000014 (20)
> > 	Name             : mtk-cam-p1 main stream
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000015   : 0: Sink
> > 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video3 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video4:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04201000
> > 		Video Capture Multiplanar
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x0300001c
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000001a (26)
> > 	Name             : mtk-cam-p1 packed out
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100001b   : 0: Sink
> > 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video4 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video5:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000022
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000020 (32)
> > 	Name             : mtk-cam-p1 partial meta 0
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000021   : 0: Sink
> > 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video5 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video6:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000028
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000026 (38)
> > 	Name             : mtk-cam-p1 partial meta 1
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000027   : 0: Sink
> > 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video6 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video7:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x0300002e
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x0000002c (44)
> > 	Name             : mtk-cam-p1 partial meta 2
> > 	Function         : V4L2 I/O
> > 	Pad 0x0100002d   : 0: Sink
> > 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video7 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/video8:
> > 
> > Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Card type        : mtk-cam-p1
> > 	Bus info         : platform:1a000000.camisp
> > 	Driver version   : 4.19.89
> > 	Capabilities     : 0x84a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > 		Device Capabilities
> > 	Device Caps      : 0x04a00000
> > 		Metadata Capture
> > 		Streaming
> > 		Extended Pix Format
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000034
> > 	Type             : V4L Video
> > Entity Info:
> > 	ID               : 0x00000032 (50)
> > 	Name             : mtk-cam-p1 partial meta 3
> > 	Function         : V4L2 I/O
> > 	Pad 0x01000033   : 0: Sink
> > 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 	test VIDIOC_QUERYCAP: OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/video8 open: OK
> > 	test VIDIOC_QUERYCAP: OK
> > 	test VIDIOC_G/S_PRIORITY: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK
> > 	test VIDIOC_TRY_FMT: OK
> > 	test VIDIOC_S_FMT: OK
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> > 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> > 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> > 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK
> > 
> > Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000050
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000001 (1)
> > 	Name             : mtk-cam-p1
> > 	Function         : Video Pixel Formatter
> > 	Pad 0x01000002   : 0: Sink
> > 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> > 	Pad 0x01000003   : 1: Source
> > 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> > 	Pad 0x01000004   : 2: Source
> > 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> > 	Pad 0x01000005   : 3: Source
> > 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> > 	Pad 0x01000006   : 4: Source
> > 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> > 	Pad 0x01000007   : 5: Source
> > 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> > 	Pad 0x01000008   : 6: Source
> > 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> > 	Pad 0x01000009   : 7: Source
> > 	Pad 0x0100000a   : 8: Source
> > 	Pad 0x0100000b   : 9: Source
> > 	Pad 0x0100000c   : 10: Source
> > 	Pad 0x0100000d   : 11: Sink
> > 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev0 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 1):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 2):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 3):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 4):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 5):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 6):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 7):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 8):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 9):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 10):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 11):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> > 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 0 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000052
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000038 (56)
> > 	Name             : 1a040000.seninf
> > 	Function         : Video Interface Bridge
> > 	Pad 0x01000039   : 0: Sink
> > 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> > 	Pad 0x0100003a   : 1: Sink
> > 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> > 	Pad 0x0100003b   : 2: Sink
> > 	Pad 0x0100003c   : 3: Sink
> > 	Pad 0x0100003d   : 4: Source
> > 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> > 	Pad 0x0100003e   : 5: Source
> > 	Pad 0x0100003f   : 6: Source
> > 	Pad 0x01000040   : 7: Source
> > 	Pad 0x01000041   : 8: Source
> > 	Pad 0x01000042   : 9: Source
> > 	Pad 0x01000043   : 10: Source
> > 	Pad 0x01000044   : 11: Source
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev1 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 1):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 2):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Sink Pad 3):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 4):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 5):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 6):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 7):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 8):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 9):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 10):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 11):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 2 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000054
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000045 (69)
> > 	Name             : ov8856 2-0010
> > 	Function         : Camera Sensor
> > 	Pad 0x01000046   : 0: Source
> > 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev2 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 0):
> > 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> > 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> > 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> > 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> > 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 11 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000056
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x00000049 (73)
> > 	Name             : dw9768 2-000c
> > 	Function         : Lens Controller
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev3 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 	test VIDIOC_G/S_CTRL: OK
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 2 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> > --------------------------------------------------------------------------------
> > Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> > 
> > Media Driver Info:
> > 	Driver name      : mtk-cam-p1
> > 	Model            : mtk-cam-p1
> > 	Serial           : 
> > 	Bus info         : platform:1a000000.camisp
> > 	Media version    : 4.19.89
> > 	Hardware revision: 0x00000000 (0)
> > 	Driver version   : 4.19.89
> > Interface Info:
> > 	ID               : 0x03000058
> > 	Type             : V4L Sub-Device
> > Entity Info:
> > 	ID               : 0x0000004a (74)
> > 	Name             : ov02a10 4-003d
> > 	Function         : Camera Sensor
> > 	Pad 0x0100004b   : 0: Source
> > 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> > 
> > Required ioctls:
> > 	test MC information (see 'Media Driver Info' above): OK
> > 
> > Allow for multiple opens:
> > 	test second /dev/v4l-subdev4 open: OK
> > 	test for unlimited opens: OK
> > 
> > Debug ioctls:
> > 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> > 
> > Input ioctls:
> > 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> > 
> > Output ioctls:
> > 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> > 
> > Input/Output configuration ioctls:
> > 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > 	test VIDIOC_G/S_EDID: OK (Not Supported)
> > 
> > Sub-Device ioctls (Source Pad 0):
> > 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> > 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
> > 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > 
> > Control ioctls:
> > 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > 	test VIDIOC_QUERYCTRL: OK
> > 		fail: v4l2-test-controls.cpp(362): returned control value out of range
> > 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> > 	test VIDIOC_G/S_CTRL: FAIL
> > 		fail: v4l2-test-controls.cpp(549): returned control value out of range
> > 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> > 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> > 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> > 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> > 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > 	Standard Controls: 10 Private Controls: 0
> > 
> > Format ioctls:
> > 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> > 	test VIDIOC_G/S_PARM: OK (Not Supported)
> > 	test VIDIOC_G_FBUF: OK (Not Supported)
> > 	test VIDIOC_G_FMT: OK (Not Supported)
> > 	test VIDIOC_TRY_FMT: OK (Not Supported)
> > 	test VIDIOC_S_FMT: OK (Not Supported)
> > 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > 	test Cropping: OK (Not Supported)
> > 	test Composing: OK (Not Supported)
> > 	test Scaling: OK (Not Supported)
> > 
> > Codec ioctls:
> > 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > 
> > Buffer ioctls:
> > 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > 	test VIDIOC_EXPBUF: OK (Not Supported)
> > 	test Requests: OK (Not Supported)
> > 
> > Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> > 
> > Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> > 
> > 
> > Jungo Lin (5):
> >   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >   media: videodev2.h: Add new boottime timestamp type
> >   media: platform: Add Mediatek ISP P1 image & meta formats
> >   media: platform: Add Mediatek ISP P1 V4L2 device driver
> > 
> >  .../bindings/media/mediatek,camisp.txt        |   83 +
> >  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >  include/uapi/linux/videodev2.h                |   41 +
> >  24 files changed, 4226 insertions(+), 1 deletion(-)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> > 
> 
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-mediatek

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2020-04-09  2:05         ` Jungo Lin
@ 2020-04-14 12:25           ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-04-14 12:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media



On 4/8/20 11:05 PM, Jungo Lin wrote:
> Hi Helen:
> 
> Thanks for your comments.
> 
> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>> Hello Jungo,
>>
>> I was taking a look at this patch (thanks for the work),
>> I didn't look in deep details, but I have some comments, please see
>> below. I hope it helps.
>>
>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>> This patch adds the Mediatek ISP P1 HW control device driver.
>>> It handles the ISP HW configuration, provides interrupt handling and
>>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
>>> implement standard V4L2 video driver that utilizes V4L2 and media
>>> framework APIs. It supports one media device, one sub-device and
>>> several video devices during initialization. Moreover, it also connects
>>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
>>> co-process via SCP communication to compose ISP registers in the
>>> firmware.
>>>
>>> (The current metadata interface used in meta input and partial
>>> meta nodes is only a temporary solution to kick off the driver
>>> development and is not ready to be reviewed yet.)
>>>
>>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
>>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
>>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
>>> ---
>>> Changes from v6:
>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>>>  - Correct auto suspend timer value for suspend/resume issue
>>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>>>  - Fix KE due to no sen-inf sub-device
>>> ---
>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>
>> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
>> capture, which for metadata, and what is being used to deal with requests.
>>
>> It would make it easier to review imho.
>>
> 
> For file structure design, it was reviewed in the previous patch
> serials.
> e.g.
> https://patchwork.kernel.org/patch/10938137/
> If you think it is better, I will modify it.

Right, I saw a suggestion to merge two files there.

I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
So it reflects a bit the topology.
But it is also up to you to see if it improves organization or not, it is just a suggestion.

> 
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>
>> It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.
>>
>> Or maybe something like:
>>
>> mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
>> mtkisp_p1_capture.c
>> mtkisp_p1_meta.c
>> mtkisp_p1_isp.c
>> mtkisp_p1_hw.c (or maybe split this between the other files)
>> mtkisp_p1_request.c
>> mtkisp_p1_common.c (?)
>>
>> or s/mtkisp_p1/mtk_cam/
>>
>> what do you think?
>>
> 
> Ok, I will revise our naming issue for consistency reason.
> 
>>>  9 files changed, 3377 insertions(+)
>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>
>>> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
>>> new file mode 100644
>>> index 000000000000..f86e1b59ad1e
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/Kconfig
>>> @@ -0,0 +1,20 @@
>>> +config VIDEO_MEDIATEK_ISP_PASS1
>>> +	tristate "Mediatek ISP Pass 1 driver"
>>> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
>>
>> I think you need OF as well
>>
>> depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF
>>
>>> +	depends on ARCH_MEDIATEK
>>
>> depends on ARCH_MEDIATEK || COMPILE_TEST
>>
> 
> Ok, we will fix this in next patch.
> 
>>> +	select V4L2_FWNODE
>>> +	select VIDEOBUF2_VMALLOC
>>> +	select VIDEOBUF2_DMA_CONTIG
>>> +	select MTK_SCP
>>> +	default n
>>> +	help
>>> +		Pass 1 driver controls 3A (auto-focus, exposure,
>>> +		and white balance) with tuning feature and outputs
>>> +		the captured image buffers in Mediatek's camera system.
>>> +
>>> +		Choose Y if you want to use Mediatek SoCs to create image
>>> +		captured application such as video recording and still image
>>> +		capturing.
>>
>> I would re-word this a bit, since people can use a captured application (and not create one) :)
>>
> 
> Ok, I will re-word as "if you want to use image captured application
> based on Mediatek SoCs for video recording and still image capturing
> functions"
> 
>>> +
>>> +		To compile this driver as a module, choose M here; the module
>>> +		will be called mtk-cam-isp.
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
>>> new file mode 100644
>>> index 000000000000..ce79d283b209
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
>>> @@ -0,0 +1,3 @@
>>> +# SPDX-License-Identifier: GPL-2.0
>>> +
>>> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
>>> \ No newline at end of file
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>> new file mode 100644
>>> index 000000000000..53b54d3c26a0
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>> @@ -0,0 +1,6 @@
>>> +# SPDX-License-Identifier: GPL-2.0
>>> +
>>> +mtk-cam-isp-objs += mtk_cam.o
>>> +mtk-cam-isp-objs += mtk_cam-hw.o
>>> +
>>> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
>>> \ No newline at end of file
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>> new file mode 100644
>>> index 000000000000..4065d0d29b7f
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>> @@ -0,0 +1,636 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +//
>>> +// Copyright (c) 2019 MediaTek Inc.
>>> +
>>> +#include <linux/atomic.h>
>>> +#include <linux/clk.h>
>>> +#include <linux/interrupt.h>
>>> +#include <linux/iopoll.h>
>>> +#include <linux/of_platform.h>
>>> +#include <linux/of_irq.h>
>>> +#include <linux/module.h>
>>> +#include <linux/remoteproc/mtk_scp.h>
>>> +#include <linux/pm_runtime.h>
>>> +#include <linux/remoteproc.h>
>>> +#include <linux/sched.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/types.h>
>>> +#include <linux/videodev2.h>
>>> +#include <linux/vmalloc.h>
>>> +
>>> +#include <media/v4l2-event.h>
>>
>> Please sort headers alphabetically.
>>
> 
> Will fix in next patch.
> 
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-hw.h"
>>> +#include "mtk_cam-regs.h"
>>> +
>>> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
>>> +#define MTK_ISP_CQ_BUFFER_COUNT			3
>>> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
>>> +
>>> +/*
>>> + *
>>> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
>>> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
>>> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
>>> + * In current driver, only supports CAM B.
>>> + *
>>> + */
>>> +#define MTK_ISP_CAM_ID_B			3
>>> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
>>> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
>>> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
>>> +
>>> +static void isp_tx_frame_worker(struct work_struct *work)
>>
>> I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
>> an external function.
>>
> 
> Fix in next patch.
> 
>>> +{
>>> +	struct mtk_cam_dev_request *req =
>>> +		container_of(work, struct mtk_cam_dev_request, frame_work);
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
>>> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
>>> +	struct device *dev = p1_dev->dev;
>>> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
>>> +
>>> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
>>> +
>>> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
>>> +		dev_err(dev, "wrong IPI len:%d\n", len);
>>> +		return;
>>> +	}
>>> +
>>> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
>>> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
>>> +		return;
>>> +
>>> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
>>> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
>>> +}
>>> +
>>> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct device *dev = p1_dev->dev;
>>> +	int ret;
>>> +
>>> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
>>> +			       isp_composer_handler, p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register IPI cmd\n");
>>> +		return ret;
>>> +	}
>>> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
>>> +			       isp_composer_handler, p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register IPI frame\n");
>>> +		goto unreg_ipi_cmd;
>>> +	}
>>> +
>>> +	p1_dev->composer_wq =
>>> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
>>> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
>>> +					WQ_FREEZABLE);
>>> +	if (!p1_dev->composer_wq) {
>>> +		dev_err(dev, "failed to alloc composer workqueue\n");
>>> +		goto unreg_ipi_frame;
>>> +	}
>>> +
>>> +	return 0;
>>> +
>>> +unreg_ipi_frame:
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
>>> +unreg_ipi_cmd:
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	destroy_workqueue(p1_dev->composer_wq);
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
>>> +}
>>> +
>>> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
>>> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
>>> +
>>> +	/*
>>> +	 * Passed coherent reserved memory info. for SCP firmware usage.
>>> +	 * This buffer is used for SCP's ISP composer to compose.
>>> +	 * The size of is fixed to 0x200000 for the requirement of composer.
>>> +	 */
>>> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
>>> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +
>>> +	isp_composer_uninit(p1_dev);
>>
>> I think you can copy the 3 lines of this isp_composer_uninit() function here, since
>> this seems the only place it is being used, and having a deinit and uninit function is
>> a bit confusing.
>>
> 
> Fix in next patch.
> 
>>> +}
>>> +
>>> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
>>> +		       struct p1_config_param *config_param)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
>>> +	memcpy(&composer_tx_cmd.config_param, config_param,
>>> +	       sizeof(*config_param));
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
>>
>> I prefer not having a int parameter, this is easier to read:
>>
>> mtk_isp_stream_on(cam);
>> mtk_isp_stream_off(cam);
>>
>> or
>>
>> mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
>> mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);
>>
>> instead of:
>>
>> mtk_isp_stream(cam, 1);
>> mtk_isp_stream(cam, 0);
>>
>> You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.
>>
> 
> Ok, I will choose the method 2.
> 
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
>>> +	composer_tx_cmd.is_stream_on = on;
>>
>> s/is_stream_on/is_streaming
>>
> 
> Fix in next patch.
> 
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	int ret;
>>> +
>>> +	ret = rproc_boot(p1_dev->rproc_handle);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to rproc_boot\n");
>>
>> It would be nice to improve this error message for users, how about:
>>
>> dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);
>>
>> Or maybe even remove this message, since rproc_boot() already have several error messages.
>>
> 
> Ok, we will remove the error message.
> 
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = isp_composer_init(p1_dev);
>>> +	if (ret)
>>
>> should rproc_shutdown() be called here?
>>
> 
> Yes, we will fix it.
> 
>>> +		return ret;
>>> +
>>> +	pm_runtime_get_sync(dev);
>>
>> You should check return value here.
>>
> 
> Fix in next patch.
> 
>>> +	isp_composer_hw_init(p1_dev);
>>> +
>>> +	p1_dev->enqueued_frame_seq_no = 0;
>>> +	p1_dev->dequeued_frame_seq_no = 0;
>>> +	p1_dev->composed_frame_seq_no = 0;
>>> +	p1_dev->sof_count = 0;
>>> +
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	isp_composer_hw_deinit(p1_dev);
>>> +	pm_runtime_mark_last_busy(dev);
>>> +	pm_runtime_put_autosuspend(dev);
>>> +	rproc_shutdown(p1_dev->rproc_handle);
>>> +
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>>> +			 struct mtk_cam_dev_request *req)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	/* Accumulated frame sequence number */
>>> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
>>> +
>>> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>>> +	queue_work(p1_dev->composer_wq, &req->frame_work);
>>> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>>> +		req->req.debug_str, req->frame_params.frame_seq_no,
>>> +		cam->running_job_count);
>>> +}
>>> +
>>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>>> +			       unsigned int dequeued_frame_seq_no)
>>> +{
>>> +	dma_addr_t base_addr = p1_dev->composer_iova;
>>> +	struct device *dev = p1_dev->dev;
>>> +	struct mtk_cam_dev_request *req;
>>> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
>>> +	unsigned int addr_offset;
>>> +
>>> +	/* Send V4L2_EVENT_FRAME_SYNC event */
>>> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>> +
>>> +	p1_dev->sof_count += 1;
>>> +	/* Save frame information */
>>> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
>>> +
>>> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>> +	if (req)
>>> +		req->timestamp = ktime_get_boottime_ns();
>>> +
>>> +	/* Update CQ base address if needed */
>>> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
>>> +		dev_dbg(dev,
>>> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
>>> +			composed_frame_seq_no, dequeued_frame_seq_no);
>>> +		return;
>>> +	}
>>> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>>> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>>> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>>> +	dev_dbg(dev,
>>> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
>>> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
>>> +}
>>> +
>>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	u32 val;
>>> +
>>> +	dev_err(p1_dev->dev,
>>> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
>>> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
>>> +	dev_err(p1_dev->dev,
>>> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
>>> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
>>
>> I think if would be better to transfor those into dev_dbg and add a counter
>> in debugfs.
>>
> 
> These error messages are important for debugging.
> I suggest to keep in dev_err.

I mean, these messages are usefull for debug (as you mentioned yourself), but for an
end user not so much, since end users won't know the meaning of those values.

For end users a "dma failure" message would be enough, then advanced users can enable
debug messages to see more.

> 
> Moreover, could you give more information about debug counter?
> I don't get your point.
> Do you suggest to accumulate the total count of DMA errors?


Yes, you could have a debugfs entry with error counters like:

cat /debugfs/mtk_isp/dma_err
8

So it is easier if this error happens very frequent or not.
In the rkisp1 case we added:

/debugfs/rkisp1/data_loss
/debugfs/rkisp1/pic_size_error
/debugfs/rkisp1/mipi_error
/debugfs/rkisp1/stats_error
/debugfs/rkisp1/mp_stop_timeout
/debugfs/rkisp1/sp_stop_timeout
/debugfs/rkisp1/mp_frame_drop
/debugfs/rkisp1/sp_frame_drop

Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.

But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).

> 
>>> +
>>> +	/* Disable DMA error mask to avoid too much error log */
>>> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
>>> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
>>> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
>>> +}
>>> +
>>> +static irqreturn_t isp_irq_cam(int irq, void *data)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
>>> +	struct device *dev = p1_dev->dev;
>>> +	unsigned int dequeued_frame_seq_no;
>>> +	unsigned int irq_status, err_status, dma_status;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
>>> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
>>> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
>>> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
>>> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
>>> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
>>> +
>>> +	/*
>>> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
>>> +	 * If these two ISRs come together, print warning msg to hint.
>>> +	 */
>>> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>>> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>>> +
>>> +	/* De-queue frame */
>>> +	if (irq_status & SW_PASS1_DON_ST) {
>>
>> I suppose this means "done streaming"?
>>
> 
> Yes, it means the frame buffer is outputed done.
> 
>>> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
>>> +					      p1_dev->dequeued_frame_seq_no);
>>> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
>>> +	}
>>> +
>>> +	/* Save frame info. & update CQ address for frame HW en-queue */
>>> +	if (irq_status & SOF_INT_ST)
>>> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
>>> +
>>> +	/* Check ISP error status */
>>> +	if (err_status) {
>>> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
>>> +		/* Show DMA errors in detail */
>>> +		if (err_status & DMA_ERR_ST)
>>> +			isp_irq_handle_dma_err(p1_dev);
>>> +	}
>>> +
>>> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
>>> +		p1_dev->sof_count, irq_status, dma_status,
>>> +		dequeued_frame_seq_no);
>>> +
>>> +	return IRQ_HANDLED;
>>> +}
>>> +
>>> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
>>> +			       struct platform_device *pdev)
>>> +{
>>> +	struct device *dev = p1_dev->dev;
>>> +	dma_addr_t addr;
>>> +	void *ptr;
>>
>> Maybe "composer_buffer" would be a better name.
>>
>> But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?
>>
> 
> Ok, I will use p1_dev->composer_virt_addr directly.
> 
>>> +	int ret;
>>> +
>>> +	p1_dev->scp = scp_get(pdev);
>>> +	if (!p1_dev->scp) {
>>> +		dev_err(dev, "failed to get scp device\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
>>> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
>>> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
>>
>> I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
>> which mapping we are refering to.
>>
> 
> Fix in next patch.
> 
>>> +
>>> +	/*
>>> +	 * Allocate coherent reserved memory for SCP firmware usage.
>>> +	 * The size of SCP composer's memory is fixed to 0x200000
>>> +	 * for the requirement of firmware.
>>> +	 */
>>> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>>> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>>> +	if (!ptr) {
>>> +		ret = -ENOMEM;
>>> +		goto fail_put_scp;
>>> +	}
>>> +
>>> +	p1_dev->composer_scp_addr = addr;
>>> +	p1_dev->composer_virt_addr = ptr;
>>> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>>> +
>>> +	/*
>>> +	 * This reserved memory is also be used by ISP P1 HW.
>>> +	 * Need to get iova address for ISP P1 DMA.
>>> +	 */
>>> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
>>> +	if (dma_mapping_error(dev, addr)) {
>>> +		dev_err(dev, "failed to map scp iova\n");
>>> +		ret = -ENOMEM;
>>> +		goto fail_free_mem;
>>> +	}
>>> +	p1_dev->composer_iova = addr;
>>
>> why not rename this to composer_isp_addr ?
>> Since, afaik, composer_scp_addr is also iova.
>>
>> At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).
>>
> 
> Ok, we will rename composer_iova to composer_isp_addr.
> Basically, scp_addr is reserved physical address and it is not behind an
> IOMMU.
> 
>>> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
>>> +
>>> +	return 0;
>>> +
>>> +fail_free_mem:
>>> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +			  p1_dev->composer_virt_addr,
>>> +			  p1_dev->composer_scp_addr);
>>> +	p1_dev->composer_scp_addr = 0;
>>> +fail_put_scp:
>>> +	scp_put(p1_dev->scp);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +			  p1_dev->composer_virt_addr,
>>> +			  p1_dev->composer_scp_addr);
>>> +	p1_dev->composer_scp_addr = 0;
>>> +	scp_put(p1_dev->scp);
>>> +}
>>> +
>>> +static int mtk_isp_pm_suspend(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	u32 val;
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "- %s\n", __func__);
>>> +
>>> +	if (pm_runtime_suspended(dev))
>>> +		return 0;
>>> +
>>> +	/* Disable ISP's view finder and wait for TG idle if possible */
>>> +	dev_dbg(dev, "cam suspend, disable VF\n");
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>>> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>>> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
>>> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>>> +
>>> +	/* Disable CMOS */
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>>> +
>>> +	/* Force ISP HW to idle */
>>> +	ret = pm_runtime_force_suspend(dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to force suspend:%d\n", ret);
>>> +		goto reenable_hw;
>>> +	}
>>> +
>>> +	return 0;
>>> +
>>> +reenable_hw:
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_isp_pm_resume(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	u32 val;
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "- %s\n", __func__);
>>> +
>>> +	if (pm_runtime_suspended(dev))
>>> +		return 0;
>>> +
>>> +	/* Force ISP HW to resume */
>>> +	ret = pm_runtime_force_resume(dev);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	/* Enable CMOS */
>>> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>>> +
>>> +	/* Enable VF */
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_runtime_suspend(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	dev_dbg(dev, "%s:disable clock\n", __func__);
>>> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_runtime_resume(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "%s:enable clock\n", __func__);
>>> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to enable clock:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_probe(struct platform_device *pdev)
>>> +{
>>> +	/* List of clocks required by isp cam */
>>> +	static const char * const clk_names[] = {
>>> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
>>> +	};
>>> +	struct mtk_isp_p1_device *p1_dev;
>>> +	struct device *dev = &pdev->dev;
>>> +	struct resource *res;
>>> +	int irq, ret, i;
>>> +
>>> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
>>> +	if (!p1_dev)
>>> +		return -ENOMEM;
>>> +
>>> +	p1_dev->dev = dev;
>>> +	dev_set_drvdata(dev, p1_dev);
>>> +
>>> +	/*
>>> +	 * Now only support single CAM with CAM B.
>>> +	 * Get CAM B register base with CAM B index.
>>> +	 * Support multiple CAMs in future.
>>> +	 */
>>> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
>>> +	p1_dev->regs = devm_ioremap_resource(dev, res);
>>> +	if (IS_ERR(p1_dev->regs)) {
>>> +		dev_err(dev, "failed to map reister base\n");
>>
>> s/reister/register
>>
> 
> Fix in next patch.
> 
>>> +		return PTR_ERR(p1_dev->regs);
>>> +	}
>>> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
>>> +
>>> +	/*
>>> +	 * The cam_sys unit only supports reg., but has no IRQ support.
>>> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
>>> +	 */
>>> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
>>> +	if (!irq) {
>>> +		dev_err(dev, "failed to get irq\n");
>>> +		return -ENODEV;
>>> +	}
>>> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
>>> +			       p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to request irq=%d\n", irq);
>>> +		return ret;
>>> +	}
>>> +	dev_dbg(dev, "registered irq=%d\n", irq);
>>> +	spin_lock_init(&p1_dev->spinlock_irq);
>>> +
>>> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
>>> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
>>> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
>>> +	if (!p1_dev->clks)
>>> +		return -ENOMEM;
>>> +
>>> +	for (i = 0; i < p1_dev->num_clks; ++i)
>>> +		p1_dev->clks[i].id = clk_names[i];
>>> +
>>> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
>>> +	pm_runtime_use_autosuspend(dev);
>>> +	pm_runtime_enable(dev);
>>> +
>>> +	/* Initialize the v4l2 common part */
>>> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
>>> +	if (ret) {
>>> +		isp_teardown_scp_rproc(p1_dev);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_remove(struct platform_device *pdev)
>>> +{
>>> +	struct device *dev = &pdev->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
>>> +	pm_runtime_dont_use_autosuspend(dev);
>>> +	pm_runtime_disable(dev);
>>> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
>>> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
>>> +			     DMA_ATTR_SKIP_CPU_SYNC);
>>> +	isp_teardown_scp_rproc(p1_dev);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static const struct dev_pm_ops mtk_isp_pm_ops = {
>>> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
>>> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
>>> +			   NULL)
>>> +};
>>> +
>>> +static const struct of_device_id mtk_isp_of_ids[] = {
>>> +	{.compatible = "mediatek,mt8183-camisp",},
>>> +	{}
>>> +};
>>> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
>>> +
>>> +static struct platform_driver mtk_isp_driver = {
>>> +	.probe   = mtk_isp_probe,
>>> +	.remove  = mtk_isp_remove,
>>> +	.driver  = {
>>> +		.name  = "mtk-cam-p1",
>>> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
>>> +		.pm     = &mtk_isp_pm_ops,
>>> +	}
>>> +};
>>> +
>>> +module_platform_driver(mtk_isp_driver);
>>> +
>>> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
>>> +MODULE_LICENSE("GPL v2");
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>> new file mode 100644
>>> index 000000000000..837662f92a5e
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>
>> This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?
>>
> 
> Ok, revise in next patch.
> 
>>> @@ -0,0 +1,64 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_HW_H__
>>> +#define __MTK_CAM_HW_H__
>>> +
>>> +#include <linux/types.h>
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-ipi.h"
>>> +
>>> +/*
>>> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
>>> + *
>>> + * @dev: Pointer to device.
>>> + * @scp_pdev: Pointer to SCP platform device.
>>> + * @rproc_handle: Pointer to new remoteproc instance.
>>> + * @cam_dev: Embedded struct cam_dev
>>> + * @regs: Camera ISP HW base register address
>>> + * @num_clks: The number of driver's clocks
>>> + * @clks: The clock data array
>>> + * @spinlock_irq: Used to protect register read/write data
>>> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
>>> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
>>> + * @composed_frame_seq_no: Frame sequence number of composed frame
>>> + * @timestamp: Frame timestamp in ns
>>> + * @sof_count: SOF counter
>>> + * @composer_wq: The work queue for frame request composing
>>> + * @composer_scp_addr: SCP address of ISP composer memory
>>> + * @composer_iova: DMA address of ISP composer memory
>>> + * @virt_addr: Virtual address of ISP composer memory
>>> + *
>>> + */
>>> +struct mtk_isp_p1_device {
>>> +	struct device *dev;
>>> +	struct mtk_scp *scp;
>>> +	struct rproc *rproc_handle;
>>> +	struct mtk_cam_dev cam_dev;
>>> +	void __iomem *regs;
>>> +	unsigned int num_clks;
>>> +	struct clk_bulk_data *clks;
>>> +	/* Used to protect register read/write data */
>>> +	spinlock_t spinlock_irq;
>>> +	unsigned int enqueued_frame_seq_no;
>>> +	unsigned int dequeued_frame_seq_no;
>>> +	unsigned int composed_frame_seq_no;
>>> +	u8 sof_count;
>>> +	struct workqueue_struct *composer_wq;
>>> +	dma_addr_t composer_scp_addr;
>>> +	dma_addr_t composer_iova;
>>> +	void *composer_virt_addr;
>>> +};
>>> +
>>> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
>>> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
>>> +		       struct p1_config_param *config_param);
>>> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
>>> +			 struct mtk_cam_dev_request *req);
>>
>> It would be nice to have docs for these too.
>>
> 
> Ok, add in next patch.
> 
>>> +
>>> +#endif /* __MTK_CAM_HW_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>> new file mode 100644
>>> index 000000000000..981b634dd91f
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>
>> I'm skipping this file, since, if I understand correctly, this is not ready for review right?
>>
> 
> I think this file is ready for review.
> 
> 
>>> @@ -0,0 +1,222 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_IPI_H__
>>> +#define __MTK_CAM_IPI_H__
>>> +
>>> +#include <linux/types.h>
>>> +
>>> +/*
>>> + * struct img_size - Image size information.
>>> + *
>>> + * @w: Image width, the unit is pixel
>>> + * @h: Image height, the unit is pixel
>>> + * @xsize: Bytes per line based on width.
>>> + * @stride: Bytes per line when changing line.
>>> + *          Stride is based on xsize + HW constrain(byte align).
>>> + *
>>> + */
>>> +struct img_size {
>>> +	u32 w;
>>> +	u32 h;
>>> +	u32 xsize;
>>> +	u32 stride;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_img_crop - image corp information
>>> + *
>>> + * @left: The left of crop area.
>>> + * @top: The top of crop area.
>>> + * @width: The width of crop area.
>>> + * @height: The height of crop area.
>>> + *
>>> + */
>>> +struct p1_img_crop {
>>> +	u32 left;
>>> +	u32 top;
>>> +	u32 width;
>>> +	u32 height;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct dma_buffer - DMA buffer address information
>>> + *
>>> + * @iova: DMA address for ISP DMA device
>>> + * @scp_addr: SCP address for external co-process unit
>>> + *
>>> + */
>>> +struct dma_buffer {
>>> +	u32 iova;
>>
>> I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).
>>
> 
> Ok, revise in next patch.
> 
>>> +	u32 scp_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_img_output - ISP P1 image output information
>>> + *
>>> + * @buffer: DMA buffer address of image.
>>> + * @size: The image size configuration.
>>> + * @crop: The crop configuration.
>>> + * @pixel_bits: The bits per image pixel.
>>> + * @img_fmt: The image format.
>>> + *
>>> + */
>>> +struct p1_img_output {
>>> +	struct dma_buffer buffer;
>>> +	struct img_size size;
>>> +	struct p1_img_crop crop;
>>> +	u8 pixel_bits;
>>> +	u32 img_fmt;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_in_param - Image input parameters structure.
>>> + *                       Normally, it comes from sensor information.
>>> + *
>>> + * @continuous: Indicate the sensor mode. Continuous or single shot.
>>> + * @subsample: Indicate to enables SOF subsample or not.
>>> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
>>> + * @data_pattern: Describe input data pattern.
>>> + * @raw_pixel_id: Bayer sequence.
>>> + * @tg_fps: The fps rate of TG (time generator).
>>> + * @img_fmt: The image format of input source.
>>> + * @p1_img_crop: The crop configuration of input source.
>>> + *
>>> + */
>>> +struct cfg_in_param {
>>> +	u8 continuous;
>>> +	u8 subsample;
>>> +	u8 pixel_mode;
>>> +	u8 data_pattern;
>>> +	u8 raw_pixel_id;
>>> +	u16 tg_fps;
>>> +	u32 img_fmt;
>>> +	struct p1_img_crop crop;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_main_out_param - The image output parameters of main stream.
>>> + *
>>> + * @bypass: Indicate this device is enabled or disabled or not.
>>> + * @pure_raw: Indicate the image path control.
>>> + *            True: pure raw
>>> + *            False: processing raw
>>> + * @pure_raw_pack: Indicate the image is packed or not.
>>> + *                 True: packed mode
>>> + *                 False: unpacked mode
>>> + * @p1_img_output: The output image information.
>>> + *
>>> + */
>>> +struct cfg_main_out_param {
>>> +	u8 bypass;
>>> +	u8 pure_raw;
>>> +	u8 pure_raw_pack;
>>> +	struct p1_img_output output;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_resize_out_param - The image output parameters of
>>> + *                               packed out stream.
>>> + *
>>> + * @bypass: Indicate this device is enabled or disabled or not.
>>> + * @p1_img_output: The output image information.
>>> + *
>>> + */
>>> +struct cfg_resize_out_param {
>>> +	u8 bypass;
>>> +	struct p1_img_output output;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_config_param - ISP P1 configuration parameters.
>>> + *
>>> + * @cfg_in_param: The Image input parameters.
>>> + * @cfg_main_param: The main output image parameters.
>>> + * @cfg_resize_out_param: The packed output image parameters.
>>> + * @enabled_dmas: The enabled DMA port information.
>>> + *
>>> + */
>>> +struct p1_config_param {
>>> +	struct cfg_in_param cfg_in_param;
>>> +	struct cfg_main_out_param cfg_main_param;
>>> +	struct cfg_resize_out_param cfg_resize_param;
>>> +	u32 enabled_dmas;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct P1_meta_frame - ISP P1 meta frame information.
>>> + *
>>> + * @enabled_dma: The enabled DMA port information.
>>> + * @vb_index: The VB2 index of meta buffer.
>>> + * @meta_addr: DMA buffer address of meta buffer.
>>> + *
>>> + */
>>> +struct P1_meta_frame {
>>> +	u32 enabled_dma;
>>> +	u32 vb_index;
>>> +	struct dma_buffer meta_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct isp_init_info - ISP P1 composer init information.
>>> + *
>>> + * @hw_module: The ISP Camera HW module ID.
>>> + * @cq_addr: The DMA address of composer memory.
>>> + *
>>> + */
>>> +struct isp_init_info {
>>> +	u8 hw_module;
>>> +	struct dma_buffer cq_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct isp_ack_info - ISP P1 IPI command ack information.
>>> + *
>>> + * @cmd_id: The IPI command ID is acked.
>>> + * @frame_seq_no: The IPI frame sequence number is acked.
>>> + *
>>> + */
>>> +struct isp_ack_info {
>>> +	u8 cmd_id;
>>> +	u32 frame_seq_no;
>>> +} __packed;
>>> +
>>> +/*
>>> + * The IPI command enumeration.
>>> + */
>>> +enum mtk_isp_scp_cmds {
>>> +	ISP_CMD_INIT,
>>> +	ISP_CMD_CONFIG,
>>> +	ISP_CMD_STREAM,
>>> +	ISP_CMD_DEINIT,
>>> +	ISP_CMD_ACK,
>>> +	ISP_CMD_FRAME_ACK,
>>> +	ISP_CMD_RESERVED,
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
>>> + *
>>> + * @cmd_id: The IPI command ID.
>>> + * @init_param: The init formation for ISP_CMD_INIT.
>>> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
>>> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
>>> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
>>> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
>>> + *
>>> + */
>>> +struct mtk_isp_scp_p1_cmd {
>>> +	u8 cmd_id;
>>> +	union {
>>> +		struct isp_init_info init_param;
>>> +		struct p1_config_param config_param;
>>> +		u32 enabled_dmas;
>>> +		struct P1_meta_frame meta_frame;
>>> +		u8 is_stream_on;
>>> +		struct isp_ack_info ack_info;
>>> +	};
>>> +} __packed;
>>> +
>>> +#endif /* __MTK_CAM_IPI_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>> new file mode 100644
>>> index 000000000000..ab2277f45fa4
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>> @@ -0,0 +1,95 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_REGS_H__
>>> +#define __MTK_CAM_REGS_H__
>>> +
>>> +/* ISP interrupt enable */
>>> +#define REG_CTL_RAW_INT_EN		0x0020
>>> +#define DMA_ERR_INT_EN			BIT(29)
>>> +
>>> +/* ISP interrupt status */
>>> +#define REG_CTL_RAW_INT_STAT		0x0024
>>> +#define VS_INT_ST			BIT(0)
>>> +#define TG_ERR_ST			BIT(4)
>>> +#define TG_GBERR_ST			BIT(5)
>>> +#define CQ_CODE_ERR_ST			BIT(6)
>>> +#define CQ_APB_ERR_ST			BIT(7)
>>> +#define CQ_VS_ERR_ST			BIT(8)
>>> +#define HW_PASS1_DON_ST			BIT(11)
>>> +#define SOF_INT_ST			BIT(12)
>>> +#define AMX_ERR_ST			BIT(15)
>>> +#define RMX_ERR_ST			BIT(16)
>>> +#define BMX_ERR_ST			BIT(17)
>>> +#define RRZO_ERR_ST			BIT(18)
>>> +#define AFO_ERR_ST			BIT(19)
>>> +#define IMGO_ERR_ST			BIT(20)
>>> +#define AAO_ERR_ST			BIT(21)
>>> +#define PSO_ERR_ST			BIT(22)
>>> +#define LCSO_ERR_ST			BIT(23)
>>> +#define BNR_ERR_ST			BIT(24)
>>> +#define LSCI_ERR_ST			BIT(25)
>>> +#define DMA_ERR_ST			BIT(29)
>>> +#define SW_PASS1_DON_ST			BIT(30)
>>> +
>>> +/* ISP interrupt 2 status */
>>> +#define REG_CTL_RAW_INT2_STAT		0x0034
>>> +#define AFO_DONE_ST			BIT(5)
>>> +#define AAO_DONE_ST			BIT(7)
>>> +
>>> +/* Configures sensor mode */
>>> +#define REG_TG_SEN_MODE			0x0230
>>> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
>>> +
>>> +/* View finder mode control */
>>> +#define REG_TG_VF_CON			0x0234
>>> +#define TG_VF_CON_VFDATA_EN		BIT(0)
>>> +
>>> +/* View finder mode control */
>>> +#define REG_TG_INTER_ST			0x026c
>>> +#define TG_CS_MASK			0x3f00
>>> +#define TG_IDLE_ST			BIT(8)
>>> +
>>> +/* IMGO error status register */
>>> +#define REG_IMGO_ERR_STAT		0x1360
>>> +/* RRZO error status register */
>>> +#define REG_RRZO_ERR_STAT		0x1364
>>> +/* AAO error status register */
>>> +#define REG_AAO_ERR_STAT		0x1368
>>> +/* AFO error status register */
>>> +#define REG_AFO_ERR_STAT		0x136c
>>> +/* LCSO error status register */
>>> +#define REG_LCSO_ERR_STAT		0x1370
>>> +/* BPCI error status register */
>>> +#define REG_BPCI_ERR_STAT		0x137c
>>> +/* LSCI error status register */
>>> +#define REG_LSCI_ERR_STAT		0x1384
>>> +/* LMVO error status register */
>>> +#define REG_LMVO_ERR_STAT		0x1390
>>> +/* FLKO error status register */
>>> +#define REG_FLKO_ERR_STAT		0x1394
>>> +/* PSO error status register */
>>> +#define REG_PSO_ERR_STAT		0x13a0
>>> +
>>> +/* CQ0 base address */
>>> +#define REG_CQ_THR0_BASEADDR		0x0198
>>> +/* Frame sequence number */
>>> +#define REG_FRAME_SEQ_NUM		0x13b8
>>> +
>>> +/* IRQ Error Mask */
>>> +#define INT_ST_MASK_CAM_ERR		( \
>>> +					TG_ERR_ST |\
>>> +					TG_GBERR_ST |\
>>> +					CQ_CODE_ERR_ST |\
>>> +					CQ_APB_ERR_ST |\
>>> +					CQ_VS_ERR_ST |\
>>> +					BNR_ERR_ST |\
>>> +					RMX_ERR_ST |\
>>> +					BMX_ERR_ST |\
>>> +					BNR_ERR_ST |\
>>> +					LSCI_ERR_ST |\
>>> +					DMA_ERR_ST)
>>> +
>>
>> I would add a common prefix all the registers in the file.
>>
>> Also, add some docs to know what those acronyms means would be nice.
>>
> 
> Ok, add this in next patch.
> 
> 
>>> +#endif	/* __MTK_CAM_REGS_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>> new file mode 100644
>>> index 000000000000..23fdb8b4abc5
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>> @@ -0,0 +1,2087 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +// Copyright (c) 2019 MediaTek Inc.
>>> +
>>> +#include <linux/device.h>
>>> +#include <linux/dma-mapping.h>
>>> +#include <linux/of.h>
>>> +#include <linux/of_graph.h>
>>> +#include <linux/of_platform.h>
>>> +#include <linux/module.h>
>>> +#include <linux/platform_device.h>
>>> +#include <linux/pm_runtime.h>
>>> +#include <linux/videodev2.h>
>>> +#include <media/media-entity.h>
>>> +#include <media/v4l2-async.h>
>>> +#include <media/v4l2-common.h>
>>> +#include <media/v4l2-event.h>
>>> +#include <media/v4l2-fwnode.h>
>>> +#include <media/v4l2-ioctl.h>
>>> +#include <media/v4l2-mc.h>
>>> +#include <media/v4l2-subdev.h>
>>> +#include <media/videobuf2-dma-contig.h>
>>
>> Please sort in alphabetical order.
>>
> 
> Fix in next patch
> 
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-hw.h"
>>> +
>>> +#define R_IMGO		BIT(0)
>>> +#define R_RRZO		BIT(1)
>>> +#define R_AAO		BIT(3)
>>> +#define R_AFO		BIT(4)
>>> +#define R_LCSO		BIT(5)
>>> +#define R_LMVO		BIT(7)
>>> +#define R_FLKO		BIT(8)
>>> +#define R_PSO		BIT(10)
>>
>> It would be nice to have better names of docs of what these means.
>>
> 
> Add in next patch
> 
>>> +
>>> +#define MTK_ISP_ONE_PIXEL_MODE		1
>>> +#define MTK_ISP_MIN_RESIZE_RATIO	6
>>> +#define MTK_ISP_MAX_RUNNING_JOBS	3
>>> +
>>> +#define MTK_CAM_CIO_PAD_SRC		4
>>> +#define MTK_CAM_CIO_PAD_SINK		11
>>> +
>>> +static inline struct mtk_cam_video_device *
>>> +file_to_mtk_cam_node(struct file *__file)
>>> +{
>>> +	return container_of(video_devdata(__file),
>>> +		struct mtk_cam_video_device, vdev);
>>> +}
>>> +
>>> +static inline struct mtk_cam_video_device *
>>> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
>>
>> no need for the underscore in __vq
>>
> 
> Revise in next patch
> 
>>> +{
>>> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
>>> +}
>>> +
>>> +static inline struct mtk_cam_dev_request *
>>> +mtk_cam_req_to_dev_req(struct media_request *__req)
>>> +{
>>> +	return container_of(__req, struct mtk_cam_dev_request, req);
>>> +}
>>> +
>>> +static inline struct mtk_cam_dev_buffer *
>>> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
>>> +{
>>> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
>>> +}
>>> +
>>> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
>>> +				 struct mtk_cam_dev_request *req,
>>> +				 enum vb2_buffer_state state)
>>> +{
>>> +	struct media_request_object *obj, *obj_prev;
>>> +	unsigned long flags;
>>> +	u64 ts_eof = ktime_get_boottime_ns();
>>> +
>>> +	if (!cam->streaming)
>>
>> s/streaming/is_streaming
>>
>> this makes a bit more intuitive of what the the boolean means.
>>
> 
> Revise in next patch
> 
>>> +		return;
>>> +
>>> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
>>> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
>>> +
>>> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
>>> +		struct vb2_buffer *vb;
>>> +		struct mtk_cam_dev_buffer *buf;
>>> +		struct mtk_cam_video_device *node;
>>> +
>>> +		if (!vb2_request_object_is_buffer(obj))
>>> +			continue;
>>> +		vb = container_of(obj, struct vb2_buffer, req_obj);
>>> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +		spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +		list_del(&buf->list);
>>> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
>>> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
>>> +			vb->timestamp = ts_eof;
>>> +		else
>>> +			vb->timestamp = req->timestamp;
>>> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>> +	}
>>> +}
>>> +
>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>> +						unsigned int frame_seq_no)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
>>> +			req->frame_params.frame_seq_no, frame_seq_no);
>>> +
>>> +		/* Match by the en-queued request number */
>>> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
>>> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +			return req;
>>> +		}
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>>> +				   unsigned int frame_seq_no)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>>> +			req->frame_params.frame_seq_no, frame_seq_no);
>>> +
>>> +		/* Match by the en-queued request number */
>>> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
>>> +			cam->running_job_count--;
>>> +			/* Pass to user space */
>>> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>>> +			list_del(&req->list);
>>> +			break;
>>> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
>>> +			cam->running_job_count--;
>>> +			/* Pass to user space for frame drop */
>>> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>>> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
>>> +				 req->frame_params.frame_seq_no);
>>
>> maybe a counter in debugfs instead of the warning.
>>
> 
> Do you mean to add counter to accumulate the total count of drop frames?

please see my comment above.

> Could we add this and also keep this warning message?

Userspace would still continue to work when this happens, not sure if it is worthy
adding a warn, I would move it to dev_dbg() instead IMHO.

> 
>>> +			list_del(&req->list);
>>> +		} else {
>>> +			break;
>>> +		}
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +}
>>> +
>>> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	dev_dbg(cam->dev, "%s\n", __func__);
>>> +
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
>>> +		list_del(&req->list);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
>>> +		list_del(&req->list);
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +}
>>> +
>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	if (!cam->streaming) {
>>> +		dev_dbg(cam->dev, "stream is off\n");
>>> +		return;
>>> +	}
>>> +
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
>>> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
>>> +			dev_dbg(cam->dev, "jobs are full\n");
>>> +			break;
>>> +		}
>>> +		cam->running_job_count++;
>>> +		list_del(&req->list);
>>> +		list_add_tail(&req->list, &cam->running_job_list);
>>
>> list_move_tail() can be used.
>>
> 
> Revised in this patch.
> 
>>> +		mtk_isp_req_enqueue(cam, req);
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +}
>>> +
>>> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_dev_req;
>>> +
>>> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
>>> +
>>> +	return &cam_dev_req->req;
>>> +}
>>> +
>>> +static void mtk_cam_req_free(struct media_request *req)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
>>> +
>>> +	kfree(cam_dev_req);
>>> +}
>>> +
>>> +static void mtk_cam_req_queue(struct media_request *req)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
>>> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
>>> +					       media_dev);
>>> +	unsigned long flags;
>>> +
>>> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
>>> +	vb2_request_queue(req);
>>> +
>>> +	/* add to pending job list */
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +
>>> +	mtk_cam_dev_req_try_queue(cam);
>>> +}
>>> +
>>> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
>>> +{
>>> +	switch (pix_fmt) {
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
>>> +		return 8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
>>> +		return 10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
>>> +		return 12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
>>> +		return 14;
>>> +	default:
>>> +		return 0;
>>> +	}
>>> +}
>>
>> which patchset are these pixel formats defined?
>> I couldn't find them in the ones you pointed.
>>
>> I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
>> so you can use v4l2_format_info() to get the number of bytes.
>>
> 
> I miss some files related to pixel format definition in this patch set.
> You could refer the old patch set for pixel format definition.
> https://patchwork.kernel.org/patch/11126055/
> 
>>> +
>>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
>>> +			     struct v4l2_pix_format_mplane *mp)
>>> +{
>>> +	unsigned int bpl, ppl;
>>
>> bytes per line and pixels per line right?
>>
> 
> Yes.
> 
>>> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>>
>> wouldn't be easier a get_pixel_bytes() function instead of bits?
>>
> 
> Sorry. I didn't get the point.
> The unit of return value is bits, not bytes.
> Do you suggest move bpl & ppl calculation into get_pixel_bits() and
> rename to get_pixel_bytes()?

Never mind, I misread it.

> 
>>> +	unsigned int width = mp->width;
>>> +
>>> +	bpl = 0;
>>> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
>>> +		/* Bayer encoding format & 2 bytes alignment */
>>> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
>>> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
>>> +		/*
>>> +		 * The FULL-G encoding format
>>> +		 * 1 G component per pixel
>>> +		 * 1 R component per 4 pixel
>>> +		 * 1 B component per 4 pixel
>>> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
>>> +		 */
>>> +		ppl = DIV_ROUND_UP(width * 6, 4);
>>> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
>>> +
>>> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
>>> +		if (pixel_bits == 10)
>>> +			bpl = ALIGN(bpl, 4);
>>> +		else
>>> +			bpl = ALIGN(bpl, 8);
>>> +	}
>>> +	/*
>>> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
>>> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
>>> +	 */
>>> +	bpl = ALIGN(bpl, 4);
>>> +
>>> +	mp->plane_fmt[0].bytesperline = bpl;
>>> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
>>> +
>>> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
>>> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
>>> +}
>>> +
>>> +static const struct v4l2_format *
>>> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
>>> +{
>>> +	int i;
>>
>> unsigned
>>
> 
> Revised in next patch.
> 
>>> +	const struct v4l2_format *dev_fmt;
>>> +
>>> +	for (i = 0; i < desc->num_fmts; i++) {
>>> +		dev_fmt = &desc->fmts[i];
>>> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
>>> +			return dev_fmt;
>>> +	}
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +/* Get the default format setting */
>>> +static void
>>> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
>>> +			     struct mtk_cam_dev_node_desc *queue_desc,
>>> +			     struct v4l2_format *dest)
>>> +{
>>> +	const struct v4l2_format *default_fmt =
>>> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
>>> +
>>> +	dest->type = queue_desc->buf_type;
>>> +
>>> +	/* Configure default format based on node type */
>>> +	if (!queue_desc->image) {
>>> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
>>> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
>>> +		return;
>>> +	}
>>> +
>>> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
>>> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
>>> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
>>> +	/* bytesperline & sizeimage calculation */
>>> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
>>> +	dest->fmt.pix_mp.num_planes = 1;
>>> +
>>> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
>>> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>> +}
>>> +
>>> +/* Utility functions */
>>> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
>>> +{
>>> +	switch (fmt) {
>>> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
>>> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
>>> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
>>> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_B;
>>> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_GB;
>>> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_GR;
>>> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
>>> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
>>> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
>>> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_R;
>>> +	default:
>>> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
>>> +	}
>>> +}
>>> +
>>> +static unsigned int get_sensor_fmt(unsigned int fmt)
>>> +{
>>> +	switch (fmt) {
>>> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
>>> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
>>> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
>>> +		return MTK_CAM_IMG_FMT_BAYER8;
>>> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
>>> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
>>> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
>>> +		return MTK_CAM_IMG_FMT_BAYER10;
>>> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
>>> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
>>> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
>>> +		return MTK_CAM_IMG_FMT_BAYER12;
>>> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
>>> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
>>> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
>>> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
>>> +		return MTK_CAM_IMG_FMT_BAYER14;
>>> +	default:
>>> +		return MTK_CAM_IMG_FMT_UNKNOWN;
>>> +	}
>>> +}
>>
>> I was wondering if it is not better to save all the media bus format
>> into a table, instead of having several swtch case statements.
>>
> 
> Ok, revise in next patch.
> 
>>> +
>>> +static unsigned int get_img_fmt(unsigned int fourcc)
>>> +{
>>> +	switch (fourcc) {
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
>>> +		return MTK_CAM_IMG_FMT_BAYER8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
>>> +		return MTK_CAM_IMG_FMT_BAYER10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
>>> +		return MTK_CAM_IMG_FMT_BAYER12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
>>> +		return MTK_CAM_IMG_FMT_BAYER14;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
>>> +	default:
>>> +		return MTK_CAM_IMG_FMT_UNKNOWN;
>>> +	}> +}
>>
>> same for the pixelformat.
>>
>> Then you can cache object with the pixelformat in the main struct.
>>
> 
> Ok, revise in next patch.
> 
>>> +
>>> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
>>> +			  struct p1_img_output *out_fmt, int sd_width,
>>> +			  int sd_height)
>>> +{
>>> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
>>> +
>>> +	/* Check output & input image size dimension */
>>> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
>>> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
>>> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
>>> +			node_id);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* Check resize ratio for resize out stream due to HW constraint */
>>> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
>>> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
>>> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
>>> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
>>> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
>>> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
>>> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
>>> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
>>> +	    !out_fmt->pixel_bits) {
>>> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
>>> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
>>> +		return -EINVAL;
>>> +	}
>>> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
>>> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
>>> +
>>> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
>>> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
>>> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
>>> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
>>> +
>>> +	out_fmt->crop.left = 0;
>>> +	out_fmt->crop.top = 0;
>>> +	out_fmt->crop.width = sd_width;
>>> +	out_fmt->crop.height = sd_height;
>>> +
>>> +	dev_dbg(cam->dev,
>>> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
>>> +		node_id, out_fmt->size.w, out_fmt->size.h,
>>> +		out_fmt->size.stride, out_fmt->size.xsize,
>>> +		out_fmt->crop.width, out_fmt->crop.height);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
>>> +{
>>> +	int i;
>>> +
>>> +	cam->enabled_count = 0;
>>> +	cam->enabled_dmas = 0;
>>> +	cam->stream_count = 0;
>>> +	cam->running_job_count = 0;
>>> +
>>> +	/* Get the enabled meta DMA ports */
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>> +		if (!cam->vdev_nodes[i].enabled)
>>> +			continue;
>>> +		cam->enabled_count++;
>>> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
>>> +	}
>>> +
>>> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
>>> +		cam->enabled_dmas);
>>> +}
>>> +
>>> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct p1_config_param config_param;
>>> +	struct cfg_in_param *cfg_in_param;
>>> +	struct v4l2_subdev_format sd_fmt;
>>> +	int sd_width, sd_height, sd_code;
>>
>> are this sd_* variables required? Can't sd_fmt be directly accessed?
>>
> 
> Ok, revised in next patch set.
> 
>>> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
>>> +	int ret;
>>> +
>>> +	/* Get sensor format configuration */
>>> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>>> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
>>> +	if (ret) {
>>> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +	sd_width = sd_fmt.format.width;
>>> +	sd_height = sd_fmt.format.height;
>>> +	sd_code = sd_fmt.format.code;
>>> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
>>> +		sd_code);
>>
>> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
>> it should be configured from userspace.
>>
> 
> Could you explain why?
> Moreover, how does configuration from user space?

IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305

If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).

Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
can be configured by userspace and part can't (which iirc should't be done in the media API).

Do you need to use v4l2_device_register_subdev_nodes() ?

Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
https://patchwork.kernel.org/cover/11463183/

which would be more appropriated if you don't want userspace to configure the whole pipeline.

> 
>>> +
>>> +	memset(&config_param, 0, sizeof(config_param));
>>> +
>>> +	/* Update cfg_in_param */
>>> +	cfg_in_param = &config_param.cfg_in_param;
>>> +	cfg_in_param->continuous = true;
>>> +	/* Fix to one pixel mode in default */
>>> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
>>> +	cfg_in_param->crop.width = sd_width;
>>> +	cfg_in_param->crop.height = sd_height;
>>> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
>>> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
>>> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
>>> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
>>> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* Update cfg_main_param */
>>> +	config_param.cfg_main_param.pure_raw = true;
>>> +	config_param.cfg_main_param.pure_raw_pack = true;
>>> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +			     &config_param.cfg_main_param.output,
>>> +			     sd_width, sd_height);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	/* Update cfg_resize_param */
>>> +	if (enabled_dma_ports & R_RRZO) {
>>> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
>>> +				     &config_param.cfg_resize_param.output,
>>> +				     sd_width, sd_height);
>>> +		if (ret)
>>> +			return ret;
>>> +	} else {
>>> +		config_param.cfg_resize_param.bypass = true;
>>> +	}
>>> +
>>> +	/* Update enabled_dmas */
>>> +	config_param.enabled_dmas = enabled_dma_ports;
>>> +	mtk_isp_hw_config(cam, &config_param);
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
>>> +				  unsigned int frame_seq_no)
>>> +{
>>> +	struct v4l2_event event = {
>>> +		.type = V4L2_EVENT_FRAME_SYNC,
>>> +		.u.frame_sync.frame_sequence = frame_seq_no,
>>> +	};
>>> +
>>> +	v4l2_event_queue(cam->subdev.devnode, &event);
>>> +}
>>> +
>>> +static struct v4l2_subdev *
>>> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
>>> +	struct device *dev = cam->dev;
>>> +	struct media_entity *entity;
>>> +	struct v4l2_subdev *sensor;
>>> +
>>> +	sensor = NULL;
>>> +	media_device_for_each_entity(entity, mdev) {
>>> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
>>> +			entity->name, entity->function, entity->stream_count);
>>> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
>>> +		    entity->stream_count) {
>>> +			sensor = media_entity_to_v4l2_subdev(entity);
>>> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
>>> +			break;
>>> +		}
>>> +	}
>>> +
>>> +	if (!sensor)
>>> +		dev_err(dev, "no seninf connected\n");
>>> +
>>> +	return sensor;
>>> +}
>>> +
>>> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!cam->seninf) {
>>> +		dev_err(dev, "no seninf connected\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	/* Get active sensor from graph topology */
>>> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
>>> +	if (!cam->sensor)
>>> +		return -ENODEV;
>>> +
>>> +	/* Seninf must stream on first */
>>> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream on %s:%d\n",
>>> +			cam->seninf->entity.name, ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream on %s:%d\n",
>>> +			cam->sensor->entity.name, ret);
>>> +		goto fail_seninf_off;
>>> +	}
>>> +
>>> +	ret = mtk_cam_dev_isp_config(cam);
>>> +	if (ret)
>>> +		goto fail_sensor_off;
>>> +
>>> +	cam->streaming = true;
>>> +	mtk_isp_stream(cam, 1);
>>> +	mtk_cam_dev_req_try_queue(cam);
>>> +	dev_dbg(dev, "streamed on Pass 1\n");
>>> +
>>> +	return 0;
>>> +
>>> +fail_sensor_off:
>>> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
>>> +fail_seninf_off:
>>> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream off %s:%d\n",
>>> +			cam->sensor->entity.name, ret);
>>> +		return -EPERM;
>>
>> Why -EPERM ?
>>
> 
> Ok, we will return ret directly.
> 
>>> +	}
>>> +
>>> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream off %s:%d\n",
>>> +			cam->seninf->entity.name, ret);
>>> +		return -EPERM;
>>> +	}
>>> +
>>> +	cam->streaming = false;
>>> +	mtk_isp_stream(cam, 0);
>>> +	mtk_isp_hw_release(cam);
>>> +
>>> +	dev_dbg(dev, "streamed off Pass 1\n");
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
>>> +{
>>> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
>>> +
>>> +	if (enable) {
>>> +		/* Align vb2_core_streamon design */
>>> +		if (cam->streaming) {
>>> +			dev_warn(cam->dev, "already streaming on\n");
>>
>> I think just dev_dbg is enough.
>>
> 
> Fix in next patch.
> 
>>> +			return 0;
>>> +		}
>>> +		return mtk_cam_cio_stream_on(cam);
>>> +	}
>>> +
>>> +	if (!cam->streaming) {
>>> +		dev_warn(cam->dev, "already streaming off\n");
>>
>> same here
>>
> 
> Fix in next patch.
> 
>>> +		return 0;
>>> +	}
>>> +	return mtk_cam_cio_stream_off(cam);
>>> +}
>>> +
>>> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
>>> +				      struct v4l2_fh *fh,
>>> +				      struct v4l2_event_subscription *sub)
>>> +{
>>> +	switch (sub->type) {
>>> +	case V4L2_EVENT_FRAME_SYNC:
>>> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
>>> +	default:
>>> +		return -EINVAL;
>>> +	}
>>> +}
>>> +
>>> +static int mtk_cam_media_link_setup(struct media_entity *entity,
>>> +				    const struct media_pad *local,
>>> +				    const struct media_pad *remote, u32 flags)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
>>> +	u32 pad = local->index;
>>> +
>>> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
>>> +		__func__, pad, remote->index, flags);
>>> +
>>> +	/*
>>> +	 * The video nodes exposed by the driver have pads indexes
>>> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
>>> +	 */
>>> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
>>> +		cam->dev_nodes[pad].enabled =
>>> +			!!(flags & MEDIA_LNK_FL_ENABLED);
>>
>> Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?
>>
> 
> Ok, revised in next patch.
> 
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct device *dev = cam->dev;
>>> +	unsigned long flags;
>>> +
>>> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
>>> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
>>> +
>>> +	/* added the buffer into the tracking list */
>>> +	spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +	list_add_tail(&buf->list, &node->buf_list);
>>> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +
>>> +	/* update buffer internal address */
>>> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
>>> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
>>
>> isn't it an issue if userspace queue two buffers for the same video device in the same request?
>>
>> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
>> will be at req->frame_params.dma_bufs[buf->node_id], no?
>>
>> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
>> buffer?
>>
>> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
>>
> 
> 1. yes, it is a issue if userspace queues two buffers for the same video
> device with the same request FD.
> 
> 2. All buffers which are belonged different to different video devices
> in the request list will be updated to req->frame_params.dma_bufs by
> buf->node_id.
> 
> 3. It is not allowed for userspace to queue partial buffers for all
> enabled video devices. If it happens, it may trigger DMA errors for this
> request.

So I guess you should implement a custom .req_validate() to enforce userspace follows this.

> 
>>> +}
>>> +
>>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_cam_dev_buffer *buf;
>>> +	dma_addr_t addr;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	buf->node_id = node->id;
>>> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
>>> +	buf->scp_addr = 0;
>>> +
>>> +	/* SCP address is only valid for meta input buffer */
>>> +	if (!node->desc.smem_alloc)
>>> +		return 0;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	/* Use coherent address to get iova address */
>>> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
>>> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
>>> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
>>> +		return -EFAULT;
>>> +	}
>>> +	buf->scp_addr = buf->daddr;
>>> +	buf->daddr = addr;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
>>> +	const struct v4l2_format *fmt = &node->vdev_fmt;
>>> +	unsigned int size;
>>> +
>>> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
>>> +		size = fmt->fmt.meta.buffersize;
>>> +	else
>>> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>> +
>>> +	if (vb2_plane_size(vb, 0) < size) {
>>> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
>>> +			vb2_plane_size(vb, 0), size);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
>>> +		if (vb2_get_plane_payload(vb, 0) != size) {
>>> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
>>> +				vb2_get_plane_payload(vb, 0), size);
>>> +			return -EINVAL;
>>> +		}
>>> +		return 0;
>>> +	}
>>> +
>>> +	v4l2_buf->field = V4L2_FIELD_NONE;
>>> +	vb2_set_plane_payload(vb, 0, size);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct mtk_cam_dev_buffer *buf;
>>> +	struct device *dev = cam->dev;
>>> +
>>> +	if (!node->desc.smem_alloc)
>>> +		return;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	dma_unmap_page_attrs(dev, buf->daddr,
>>> +			     vb->planes[0].length,
>>> +			     DMA_BIDIRECTIONAL,
>>> +			     DMA_ATTR_SKIP_CPU_SYNC);
>>> +}
>>> +
>>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +
>>> +	dev_dbg(cam->dev, "%s\n", __func__);
>>> +}
>>> +
>>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
>>> +				   unsigned int *num_buffers,
>>> +				   unsigned int *num_planes,
>>> +				   unsigned int sizes[],
>>> +				   struct device *alloc_devs[])
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	unsigned int max_buffer_count = node->desc.max_buf_count;
>>> +	const struct v4l2_format *fmt = &node->vdev_fmt;
>>> +	unsigned int size;
>>> +
>>> +	/* Check the limitation of buffer size */
>>> +	if (max_buffer_count)
>>> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
>>> +
>>> +	if (node->desc.smem_alloc)
>>> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>>> +
>>> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
>>> +		size = fmt->fmt.meta.buffersize;
>>> +	else
>>> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>> +
>>> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
>>> +	if (*num_planes) {
>>> +		if (sizes[0] < size || *num_planes != 1)
>>> +			return -EINVAL;
>>> +	} else {
>>> +		*num_planes = 1;
>>> +		sizes[0] = size;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
>>> +					   struct mtk_cam_video_device *node,
>>> +					   enum vb2_buffer_state state)
>>> +{
>>> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
>>> +		list_del(&buf->list);
>>> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>> +	}
>>> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +}
>>> +
>>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
>>> +				       unsigned int count)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!node->enabled) {
>>> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
>>> +		ret = -ENOLINK;
>>> +		goto fail_ret_buf;
>>> +	}
>>> +
>>> +	mutex_lock(&cam->op_lock);
>>> +	/* Start streaming of the whole pipeline now*/
>>> +	if (!cam->pipeline.streaming_count) {
>>
>> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
>>
> 
> The check is designed to start the media pipeline when we start
> streaming on the first node. You could refer the detail in below link.
> 
> https://patchwork.kernel.org/patch/10985819/

right, ok, this is when enabling streaming from multiple nodes.

media_pipeline_start() is usually called for every stream that starts.

So cam->pipeline.streaming_count can reflect the number of streams enabled.

So maybe you don't need cam->stream_count.

> 
> 
>>> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
>>> +		if (ret) {
>>> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
>>> +			goto fail_unlock;
>>> +		}
>>> +		mtk_cam_dev_init_stream(cam);
>>> +		ret = mtk_isp_hw_init(cam);

Would it make sense to move this to s_stream in the ISP's subdevice ?

>>> +		if (ret) {
>>> +			dev_err(dev, "failed to init HW:%d\n", ret);
>>> +			goto fail_stop_pipeline;
>>> +		}
>>> +	}
>>> +
>>> +	/* Media links are fixed after media_pipeline_start */
>>> +	cam->stream_count++;
>>> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
>>> +		cam->enabled_count);
>>> +	if (cam->stream_count < cam->enabled_count) {

I'm also wondering, since you need to wait for all the enabled video devices
to start streaming, shouldn't this be done inside a request? So you can enable
all of them at once?

Also, like this you wouldn't need to check enabled links to query for enabled video
nodes, you can just enable the ones in the request.

make sense?

>>> +		mutex_unlock(&cam->op_lock);
>>> +		return 0;
>>> +	}
>>> +
>>> +	/* Stream on sub-devices node */
>>> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
>>> +	if (ret)
>>> +		goto fail_no_stream;
>>> +	mutex_unlock(&cam->op_lock);
>>> +
>>> +	return 0;
>>> +
>>> +fail_no_stream:
>>> +	cam->stream_count--;
>>> +fail_stop_pipeline:
>>> +	if (cam->stream_count == 0)
>>> +		media_pipeline_stop(&node->vdev.entity);
>>> +fail_unlock:
>>> +	mutex_unlock(&cam->op_lock);
>>> +fail_ret_buf:
>>> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	struct device *dev = cam->dev;
>>> +
>>> +	mutex_lock(&cam->op_lock);
>>> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
>>> +		cam->stream_count);
>>> +	/* Check the first node to stream-off */
>>> +	if (cam->stream_count == cam->enabled_count)
>>> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
>>> +
>>> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>>> +	cam->stream_count--;
>>> +	if (cam->stream_count) {
>>> +		mutex_unlock(&cam->op_lock);
>>> +		return;
>>> +	}
>>> +	mutex_unlock(&cam->op_lock);
>>> +
>>> +	mtk_cam_dev_req_cleanup(cam);
>>> +	media_pipeline_stop(&node->vdev.entity);
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
>>> +				   struct v4l2_capability *cap)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +
>>> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
>>> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
>>> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
>>> +		 dev_name(cam->dev));
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
>>> +				   struct v4l2_fmtdesc *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (f->index >= node->desc.num_fmts)
>>> +		return -EINVAL;
>>> +
>>> +	/* f->description is filled in v4l_fill_fmtdesc function */
>>> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
>>> +	f->flags = 0;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
>>> +				struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	f->fmt = node->vdev_fmt.fmt;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
>>> +				  struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +	struct device *dev = cam->dev;
>>> +	const struct v4l2_format *dev_fmt;
>>> +	struct v4l2_format try_fmt;
>>> +
>>> +	memset(&try_fmt, 0, sizeof(try_fmt));
>>> +	try_fmt.type = f->type;
>>> +
>>> +	/* Validate pixelformat */
>>> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
>>> +	if (!dev_fmt) {
>>> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
>>> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
>>> +	}
>>> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
>>> +
>>> +	/* Validate image width & height range */
>>> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
>>> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
>>> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
>>> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
>>> +	/* 4 bytes alignment for width */
>>> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
>>> +
>>> +	/* Only support one plane */
>>> +	try_fmt.fmt.pix_mp.num_planes = 1;
>>> +
>>> +	/* bytesperline & sizeimage calculation */
>>> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
>>> +
>>> +	/* Constant format fields */
>>> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
>>> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>> +
>>> +	*f = try_fmt;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
>>> +				struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (vb2_is_busy(node->vdev.queue)) {
>>> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
>>> +		return -EBUSY;
>>> +	}
>>> +
>>> +	/* Get the valid format */
>>> +	mtk_cam_vidioc_try_fmt(file, fh, f);
>>> +	/* Configure to video device */
>>> +	node->vdev_fmt = *f;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
>>> +					  struct v4l2_frmsizeenum *sizes)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
>>> +	const struct v4l2_format *dev_fmt;
>>> +
>>> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
>>> +	if (!dev_fmt || sizes->index)
>>> +		return -EINVAL;
>>> +
>>> +	sizes->type = node->desc.frmsizes->type;
>>> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
>>> +	       sizeof(sizes->stepwise));
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
>>> +					struct v4l2_fmtdesc *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (f->index)
>>> +		return -EINVAL;
>>> +
>>> +	/* f->description is filled in v4l_fill_fmtdesc function */
>>> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
>>> +	f->flags = 0;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
>>> +				     struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
>>> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
>>> +	.subscribe_event = mtk_cam_sd_subscribe_event,
>>> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
>>> +};
>>> +
>>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
>>> +	.s_stream =  mtk_cam_sd_s_stream,
>>> +};
>>> +
>>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
>>> +	.core = &mtk_cam_subdev_core_ops,
>>> +	.video = &mtk_cam_subdev_video_ops,
>>> +};
>>
>> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
>> I wonder if pad ops shouldn't be implemented too (to be verified).
>>
> 
> Ok, I will investigate this.
> 
>>> +
>>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
>>> +	.link_setup = mtk_cam_media_link_setup,
>>> +	.link_validate = v4l2_subdev_link_validate,
>>> +};
>>> +
>>> +static const struct vb2_ops mtk_cam_vb2_ops = {
>>> +	.queue_setup = mtk_cam_vb2_queue_setup,
>>> +	.wait_prepare = vb2_ops_wait_prepare,
>>> +	.wait_finish = vb2_ops_wait_finish,
>>> +	.buf_init = mtk_cam_vb2_buf_init,
>>> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
>>> +	.start_streaming = mtk_cam_vb2_start_streaming,
>>> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
>>> +	.buf_queue = mtk_cam_vb2_buf_queue,
>>> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
>>> +	.buf_request_complete = mtk_cam_vb2_request_complete,
>>> +};> +
>>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
>>> +	.unlocked_ioctl = video_ioctl2,
>>> +	.open = v4l2_fh_open,
>>> +	.release = vb2_fop_release,
>>> +	.poll = vb2_fop_poll,
>>> +	.mmap = vb2_fop_mmap,
>>> +#ifdef CONFIG_COMPAT
>>> +	.compat_ioctl32 = v4l2_compat_ioctl32,
>>> +#endif
>>> +};
>>> +
>>> +static const struct media_device_ops mtk_cam_media_ops = {
>>> +	.req_alloc = mtk_cam_req_alloc,
>>> +	.req_free = mtk_cam_req_free,
>>> +	.req_validate = vb2_request_validate,
>>> +	.req_queue = mtk_cam_req_queue,
>>> +};
>>> +
>>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
>>> +				  struct media_device *media_dev)
>>> +{
>>> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
>>> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
>>> +	struct device *dev = cam->dev;
>>> +	int i, ret;
>>> +
>>> +	media_dev->dev = cam->dev;
>>> +	strscpy(media_dev->model, dev_driver_string(dev),
>>> +		sizeof(media_dev->model));
>>> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
>>> +		 "platform:%s", dev_name(dev));
>>> +	media_dev->hw_revision = 0;
>>> +	media_device_init(media_dev);
>>> +	media_dev->ops = &mtk_cam_media_ops;
>>> +
>>> +	ret = media_device_register(media_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register media device:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	/* Initialize subdev pads */
>>> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
>>> +					sizeof(*cam->subdev_pads),
>>> +					GFP_KERNEL);
>>> +	if (!cam->subdev_pads) {
>>> +		dev_err(dev, "failed to allocate subdev_pads\n");
>>> +		ret = -ENOMEM;
>>> +		goto fail_media_unreg;
>>> +	}
>>> +
>>> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
>>> +				     cam->subdev_pads);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
>>> +		goto fail_media_unreg;
>>> +	}
>>> +
>>> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
>>> +	for (i = 0; i < num_pads; i++)
>>> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>> +
>>> +	/* Customize the last one pad as CIO sink pad. */
>>> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>> +
>>> +	return 0;
>>> +
>>> +fail_media_unreg:
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int
>>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
>>> +			      struct mtk_cam_video_device *node)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct video_device *vdev = &node->vdev;
>>> +	struct vb2_queue *vbq = &node->vbq;
>>> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
>>> +	unsigned int link_flags = node->desc.link_flags;
>>> +	int ret;
>>> +
>>> +	/* Initialize mtk_cam_video_device */
>>> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
>>> +		node->enabled = true;
>>> +	else
>>> +		node->enabled = false;
>>> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
>>> +
>>> +	cam->subdev_pads[node->id].flags = output ?
>>> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>> +
>>> +	/* Initialize media entities */
>>> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>> +
>>> +	/* Initialize vbq */
>>> +	vbq->type = node->desc.buf_type;
>>> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
>>> +		vbq->io_modes = VB2_MMAP;
>>> +	else
>>> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
>>> +
>>> +	if (node->desc.smem_alloc) {
>>> +		vbq->bidirectional = 1;
>>> +		vbq->dev = cam->smem_dev;
>>> +	} else {
>>> +		vbq->dev = dev;
>>> +	}
>>> +	vbq->ops = &mtk_cam_vb2_ops;
>>> +	vbq->mem_ops = &vb2_dma_contig_memops;
>>> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
>>> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
>>> +	if (output)
>>> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
>>> +	else
>>> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>>> +	/* No minimum buffers limitation */
>>> +	vbq->min_buffers_needed = 0;
>>> +	vbq->drv_priv = cam;
>>> +	vbq->lock = &node->vdev_lock;
>>> +	vbq->supports_requests = true;
>>> +	vbq->requires_requests = true;
>>> +
>>> +	ret = vb2_queue_init(vbq);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
>>> +		goto fail_media_clean;
>>> +	}
>>> +
>>> +	/* Initialize vdev */
>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
>>> +		 dev_driver_string(dev), node->desc.name);
>>> +	/* set cap/type/ioctl_ops of the video device */
>>> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
>>> +	vdev->ioctl_ops = node->desc.ioctl_ops;
>>> +	vdev->fops = &mtk_cam_v4l2_fops;
>>> +	vdev->release = video_device_release_empty;
>>> +	vdev->lock = &node->vdev_lock;
>>> +	vdev->v4l2_dev = &cam->v4l2_dev;
>>> +	vdev->queue = &node->vbq;
>>> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
>>> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
>>> +	vdev->entity.ops = NULL;
>>> +	video_set_drvdata(vdev, cam);
>>> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
>>> +
>>> +	/* Initialize miscellaneous variables */
>>> +	mutex_init(&node->vdev_lock);
>>> +	INIT_LIST_HEAD(&node->buf_list);
>>> +	spin_lock_init(&node->buf_list_lock);
>>> +
>>> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register vde:%d\n", ret);
>>> +		goto fail_vb2_rel;
>>> +	}
>>> +
>>> +	/* Create link between video node and the subdev pad */
>>> +	if (output) {
>>> +		ret = media_create_pad_link(&vdev->entity, 0,
>>> +					    &cam->subdev.entity,
>>> +					    node->id, link_flags);
>>> +	} else {
>>> +		ret = media_create_pad_link(&cam->subdev.entity,
>>> +					    node->id, &vdev->entity, 0,
>>> +					    link_flags);
>>> +	}
>>
>> No need for the curly braces.
>>
> 
> Revised in next patch.
> 
>>> +	if (ret)
>>> +		goto fail_vdev_ureg;
>>> +
>>> +	return 0;
>>> +
>>> +fail_vdev_ureg:
>>> +	video_unregister_device(vdev);
>>> +fail_vb2_rel:
>>> +	mutex_destroy(&node->vdev_lock);
>>> +	vb2_queue_release(vbq);
>>> +fail_media_clean:
>>> +	media_entity_cleanup(&vdev->entity);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void
>>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
>>> +{
>>> +	video_unregister_device(&node->vdev);
>>> +	vb2_queue_release(&node->vbq);
>>> +	media_entity_cleanup(&node->vdev.entity);
>>> +	mutex_destroy(&node->vdev_lock);
>>> +}
>>> +
>>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int i, ret;
>>> +
>>> +	/* Set up media device & pads */
>>> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
>>> +	if (ret)
>>> +		return ret;
>>> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
>>> +
>>> +	/* Set up v4l2 device */
>>> +	cam->v4l2_dev.mdev = &cam->media_dev;
>>> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
>>> +		goto fail_media_unreg;
>>> +	}
>>> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
>>> +
>>> +	/* Initialize subdev */
>>> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
>>> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>>> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
>>> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>>> +				V4L2_SUBDEV_FL_HAS_EVENTS;
>>> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
>>> +		 "%s", dev_driver_string(dev));
>>> +	v4l2_set_subdevdata(&cam->subdev, cam);
>>> +
>>> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
>>> +		goto fail_clean_media_entiy;
>>> +	}
>>> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
>>> +
>>> +	/* Create video nodes and links */
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
>>> +
>>> +		node->id = node->desc.id;
>>> +		ret = mtk_cam_video_register_device(cam, node);
>>> +		if (ret)
>>> +			goto fail_vdev_unreg;
>>> +	}
>>> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
>>> +
>>> +	return 0;
>>> +
>>> +fail_vdev_unreg:
>>> +	for (i--; i >= 0; i--)
>>> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>> +fail_clean_media_entiy:
>>> +	media_entity_cleanup(&cam->subdev.entity);
>>> +	v4l2_device_unregister(&cam->v4l2_dev);
>>> +fail_media_unreg:
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
>>> +{
>>> +	int i;
>>> +
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
>>> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>> +
>>> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
>>> +	v4l2_device_unregister_subdev(&cam->subdev);
>>> +	v4l2_device_unregister(&cam->v4l2_dev);
>>> +	media_entity_cleanup(&cam->subdev.entity);
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
>>> +				      struct v4l2_subdev *sd,
>>> +				      struct v4l2_async_subdev *asd)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +
>>> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>>> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	cam->seninf = sd;
>>> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
>>> +					struct v4l2_subdev *sd,
>>> +					struct v4l2_async_subdev *asd)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +
>>> +	cam->seninf = NULL;
>>> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
>>> +}
>>> +
>>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!cam->seninf) {
>>> +		dev_err(dev, "No seninf subdev\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
>>> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
>>> +				    MEDIA_LNK_FL_IMMUTABLE |
>>> +				    MEDIA_LNK_FL_ENABLED);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
>>> +			cam->seninf->entity.name, cam->subdev.entity.name,
>>> +			ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
>>> +	.bound = mtk_cam_dev_notifier_bound,
>>> +	.unbind = mtk_cam_dev_notifier_unbind,
>>> +	.complete = mtk_cam_dev_notifier_complete,
>>> +};
>>> +
>>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	v4l2_async_notifier_init(&cam->notifier);
>>> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
>>> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
>>
>> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
>>
>> Regards,
>> Helen
>>
> 
> Ok, we will investigate how to do.
> 
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
>>> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
>>> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
>>> +		v4l2_async_notifier_cleanup(&cam->notifier);
>>> +	}
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
>>> +{
>>> +	v4l2_async_notifier_unregister(&cam->notifier);
>>> +	v4l2_async_notifier_cleanup(&cam->notifier);
>>> +}
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
>>> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
>>> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
>>> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
>>> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>> +};
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
>>> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +};
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
>>> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +};> +
>>> +static const struct v4l2_format meta_fmts[] = {
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
>>> +			.buffersize = 512 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_3A,
>>> +			.buffersize = 1200 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_AF,
>>> +			.buffersize = 640 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
>>> +			.buffersize = 288 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
>>> +			.buffersize = 256,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct v4l2_format stream_out_fmts[] = {
>>> +	/* This is a default image format */
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct v4l2_format bin_out_fmts[] = {
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct
>>> +mtk_cam_dev_node_desc output_queues[] = {
>>> +	{
>>> +		.id = MTK_CAM_P1_META_IN_0,
>>> +		.name = "meta input",
>>> +		.cap = V4L2_CAP_META_OUTPUT,
>>> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = true,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 0,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
>>> +	},
>>> +};
>>> +
>>> +static const struct
>>> +mtk_cam_dev_node_desc capture_queues[] = {
>>> +	{
>>> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +		.name = "main stream",
>>> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
>>> +		.image = true,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_IMGO,
>>> +		.fmts = stream_out_fmts,
>>> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
>>> +		.default_fmt_idx = 0,
>>> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>> +		.frmsizes = &(struct v4l2_frmsizeenum) {
>>> +			.index = 0,
>>> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>> +			.stepwise = {
>>> +				.max_width = IMG_MAX_WIDTH,
>>> +				.min_width = IMG_MIN_WIDTH,
>>> +				.max_height = IMG_MAX_HEIGHT,
>>> +				.min_height = IMG_MIN_HEIGHT,
>>> +				.step_height = 1,
>>> +				.step_width = 1,
>>> +			},
>>> +		},
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
>>> +		.name = "packed out",
>>> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>> +		.link_flags = 0,
>>> +		.image = true,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_RRZO,
>>> +		.fmts = bin_out_fmts,
>>> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
>>> +		.default_fmt_idx = 0,
>>> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>> +		.frmsizes = &(struct v4l2_frmsizeenum) {
>>> +			.index = 0,
>>> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>> +			.stepwise = {
>>> +				.max_width = IMG_MAX_WIDTH,
>>> +				.min_width = IMG_MIN_WIDTH,
>>> +				.max_height = IMG_MAX_HEIGHT,
>>> +				.min_height = IMG_MIN_HEIGHT,
>>> +				.step_height = 1,
>>> +				.step_width = 1,
>>> +			},
>>> +		},
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_0,
>>> +		.name = "partial meta 0",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_AAO | R_FLKO | R_PSO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 1,
>>> +		.max_buf_count = 5,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_1,
>>> +		.name = "partial meta 1",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_AFO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 2,
>>> +		.max_buf_count = 5,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_2,
>>> +		.name = "partial meta 2",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_LCSO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 3,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_3,
>>> +		.name = "partial meta 3",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_LMVO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 4,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +};
>>> +
>>> +/* The helper to configure the device context */
>>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
>>> +{
>>> +	unsigned int node_idx;
>>> +	int i;
>>> +
>>> +	node_idx = 0;
>>> +	/* Setup the output queue */
>>> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
>>> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
>>> +
>>> +	/* Setup the capture queue */
>>> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
>>> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
>>> +}
>>> +
>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>> +		     struct mtk_cam_dev *cam)
>>> +{
>>> +	int ret;
>>> +
>>> +	cam->dev = &pdev->dev;
>>> +	mtk_cam_dev_queue_setup(cam);
>>> +
>>> +	spin_lock_init(&cam->pending_job_lock);
>>> +	spin_lock_init(&cam->running_job_lock);
>>> +	INIT_LIST_HEAD(&cam->pending_job_list);
>>> +	INIT_LIST_HEAD(&cam->running_job_list);
>>> +	mutex_init(&cam->op_lock);
>>> +
>>> +	/* v4l2 sub-device registration */
>>> +	ret = mtk_cam_v4l2_register(cam);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	ret = mtk_cam_v4l2_async_register(cam);
>>> +	if (ret)
>>> +		goto fail_v4l2_unreg;
>>> +
>>> +	return 0;
>>> +
>>> +fail_v4l2_unreg:
>>> +	mutex_destroy(&cam->op_lock);
>>> +	mtk_cam_v4l2_unregister(cam);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
>>> +{
>>> +	mtk_cam_v4l2_async_unregister(cam);
>>> +	mtk_cam_v4l2_unregister(cam);
>>> +	mutex_destroy(&cam->op_lock);
>>> +}
>>> +
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>> new file mode 100644
>>> index 000000000000..0a340a1e65ea
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>> @@ -0,0 +1,244 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_H__
>>> +#define __MTK_CAM_H__
>>> +
>>> +#include <linux/device.h>
>>> +#include <linux/types.h>
>>> +#include <linux/platform_device.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/videodev2.h>
>>> +#include <media/v4l2-device.h>
>>> +#include <media/v4l2-ctrls.h>
>>> +#include <media/v4l2-subdev.h>
>>> +#include <media/videobuf2-core.h>
>>> +#include <media/videobuf2-v4l2.h>
>>> +
>>> +#include "mtk_cam-ipi.h"
>>> +
>>> +#define IMG_MAX_WIDTH		5376
>>> +#define IMG_MAX_HEIGHT		4032
>>> +#define IMG_MIN_WIDTH		80
>>> +#define IMG_MIN_HEIGHT		60
>>> +
>>> +/*
>>> + * ID enum value for struct mtk_cam_dev_node_desc:id
>>> + * or mtk_cam_video_device:id
>>> + */
>>> +enum  {
>>> +	MTK_CAM_P1_META_IN_0 = 0,
>>> +	MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +	MTK_CAM_P1_PACKED_BIN_OUT,
>>> +	MTK_CAM_P1_META_OUT_0,
>>> +	MTK_CAM_P1_META_OUT_1,
>>> +	MTK_CAM_P1_META_OUT_2,
>>> +	MTK_CAM_P1_META_OUT_3,
>>> +	MTK_CAM_P1_TOTAL_NODES
>>> +};
>>> +
>>> +/* Supported image format list */
>>> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
>>> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
>>> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
>>> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
>>> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
>>> +
>>> +/* Supported bayer pixel order */
>>> +#define MTK_CAM_RAW_PXL_ID_B		0
>>> +#define MTK_CAM_RAW_PXL_ID_GB		1
>>> +#define MTK_CAM_RAW_PXL_ID_GR		2
>>> +#define MTK_CAM_RAW_PXL_ID_R		3
>>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
>>> +
>>> +/*
>>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
>>> + *
>>> + * @frame_seq_no: The frame sequence of frame in driver layer.
>>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
>>> + *
>>> + */
>>> +struct mtk_p1_frame_param {
>>> +	unsigned int frame_seq_no;
>>> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_request - MTK camera device request.
>>> + *
>>> + * @req: Embedded struct media request.
>>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
>>> + * @frame_work: work queue entry for frame transmission to SCP.
>>> + * @list: List entry of the object for @struct mtk_cam_dev:
>>> + *        pending_job_list or running_job_list.
>>> + * @timestamp: Start of frame timestamp in ns
>>> + *
>>> + */
>>> +struct mtk_cam_dev_request {
>>> +	struct media_request req;
>>> +	struct mtk_p1_frame_param frame_params;
>>> +	struct work_struct frame_work;
>>> +	struct list_head list;
>>> +	u64 timestamp;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
>>> + *
>>> + * @vbb: Embedded struct vb2_v4l2_buffer.
>>> + * @list: List entry of the object for @struct mtk_cam_video_device:
>>> + *        buf_list.
>>> + * @daddr: The DMA address of this buffer.
>>> + * @scp_addr: The SCP address of this buffer which
>>> + *            is only supported for meta input node.
>>> + * @node_id: The vidoe node id which this buffer belongs to.
>>> + *
>>> + */
>>> +struct mtk_cam_dev_buffer {
>>> +	struct vb2_v4l2_buffer vbb;
>>> +	struct list_head list;
>>> +	/* Intenal part */
>>> +	dma_addr_t daddr;
>>> +	dma_addr_t scp_addr;
>>> +	unsigned int node_id;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
>>> + *
>>> + * @id: id of the node
>>> + * @name: name of the node
>>> + * @cap: supported V4L2 capabilities
>>> + * @buf_type: supported V4L2 buffer type
>>> + * @dma_port: the dma ports associated to the node
>>> + * @link_flags: default media link flags
>>> + * @smem_alloc: using the smem_dev as alloc device or not
>>> + * @image: true for image node, false for meta node
>>> + * @num_fmts: the number of supported node formats
>>> + * @default_fmt_idx: default format of this node
>>> + * @max_buf_count: maximum VB2 buffer count
>>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
>>> + * @fmts: supported format
>>> + * @frmsizes: supported V4L2 frame size number
>>> + *
>>> + */
>>> +struct mtk_cam_dev_node_desc {
>>> +	u8 id;
>>> +	const char *name;
>>> +	u32 cap;
>>> +	u32 buf_type;
>>> +	u32 dma_port;
>>> +	u32 link_flags;
>>> +	u8 smem_alloc:1;
>>> +	u8 image:1;
>>> +	u8 num_fmts;
>>> +	u8 default_fmt_idx;
>>> +	u8 max_buf_count;
>>> +	const struct v4l2_ioctl_ops *ioctl_ops;
>>> +	const struct v4l2_format *fmts;
>>> +	const struct v4l2_frmsizeenum *frmsizes;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_video_device - Mediatek video device structure
>>> + *
>>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
>>> + * @enabled: Indicate the video device is enabled or not
>>> + * @desc: The node description of video device
>>> + * @vdev_fmt: The V4L2 format of video device
>>> + * @vdev_pad: The media pad graph object of video device
>>> + * @vbq: A videobuf queue of video device
>>> + * @vdev: The video device instance
>>> + * @vdev_lock: Serializes vb2 queue and video device operations
>>> + * @buf_list: List for enqueue buffers
>>> + * @buf_list_lock: Lock used to protect buffer list.
>>> + *
>>> + */
>>> +struct mtk_cam_video_device {
>>> +	unsigned int id;
>>> +	unsigned int enabled;
>>> +	struct mtk_cam_dev_node_desc desc;
>>> +	struct v4l2_format vdev_fmt;
>>> +	struct media_pad vdev_pad;
>>> +	struct vb2_queue vbq;
>>> +	struct video_device vdev;
>>> +	/* Serializes vb2 queue and video device operations */
>>> +	struct mutex vdev_lock;
>>> +	struct list_head buf_list;
>>> +	/* Lock used to protect buffer list */
>>> +	spinlock_t buf_list_lock;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev - Mediatek camera device structure.
>>> + *
>>> + * @dev: Pointer to device.
>>> + * @smem_pdev: Pointer to shared memory device.
>>> + * @pipeline: Media pipeline information.
>>> + * @media_dev: Media device instance.
>>> + * @subdev: The V4L2 sub-device instance.
>>> + * @v4l2_dev: The V4L2 device driver instance.
>>> + * @notifier: The v4l2_device notifier data.
>>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
>>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
>>> + * @seninf: Pointer to the seninf sub-device.
>>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
>>> + * @streaming: Indicate the overall streaming status is on or off.
>>> + * @enabled_dmas: The enabled dma port information when streaming on.
>>> + * @enabled_count: Number of enabled video nodes
>>> + * @stream_count: Number of streaming video nodes
>>> + * @running_job_count: Nunber of running jobs in the HW driver.
>>> + * @pending_job_list: List to keep the media requests before en-queue into
>>> + *                    HW driver.
>>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
>>> + * @running_job_list: List to keep the media requests after en-queue into
>>> + *                    HW driver.
>>> + * @running_job_lock: Protect the running_job_list data.
>>> + * @op_lock: Serializes driver's VB2 callback operations.
>>> + *
>>> + */
>>> +struct mtk_cam_dev {
>>> +	struct device *dev;
>>> +	struct device *smem_dev;
>>> +	struct media_pipeline pipeline;
>>> +	struct media_device media_dev;
>>> +	struct v4l2_subdev subdev;
>>> +	struct v4l2_device v4l2_dev;
>>> +	struct v4l2_async_notifier notifier;
>>> +	struct media_pad *subdev_pads;
>>> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
>>> +	struct v4l2_subdev *seninf;
>>> +	struct v4l2_subdev *sensor;
>>> +	unsigned int streaming;
>>> +	unsigned int enabled_dmas;
>>> +	unsigned int enabled_count;
>>> +	unsigned int stream_count;
>>> +	unsigned int running_job_count;
>>> +	struct list_head pending_job_list;
>>> +	/* Protect the pending_job_list data */
>>> +	spinlock_t pending_job_lock;
>>> +	struct list_head running_job_list;
>>> +	/* Protect the running_job_list data & running_job_count */
>>> +	spinlock_t running_job_lock;
>>> +	/* Serializes driver's VB2 callback operations */
>>> +	struct mutex op_lock;
>>> +};
>>> +
>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>> +		     struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
>>> +				   unsigned int frame_seq_no);
>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
>>> +				  unsigned int frame_seq_no);
>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>> +						unsigned int frame_seq_no);
>>> +
>>> +#endif /* __MTK_CAM_H__ */
>>>
>>
>> _______________________________________________
>> Linux-mediatek mailing list
>> Linux-mediatek@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 

Regards,
Helen

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-04-14 12:25           ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-04-14 12:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport



On 4/8/20 11:05 PM, Jungo Lin wrote:
> Hi Helen:
> 
> Thanks for your comments.
> 
> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>> Hello Jungo,
>>
>> I was taking a look at this patch (thanks for the work),
>> I didn't look in deep details, but I have some comments, please see
>> below. I hope it helps.
>>
>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>> This patch adds the Mediatek ISP P1 HW control device driver.
>>> It handles the ISP HW configuration, provides interrupt handling and
>>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
>>> implement standard V4L2 video driver that utilizes V4L2 and media
>>> framework APIs. It supports one media device, one sub-device and
>>> several video devices during initialization. Moreover, it also connects
>>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
>>> co-process via SCP communication to compose ISP registers in the
>>> firmware.
>>>
>>> (The current metadata interface used in meta input and partial
>>> meta nodes is only a temporary solution to kick off the driver
>>> development and is not ready to be reviewed yet.)
>>>
>>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
>>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
>>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
>>> ---
>>> Changes from v6:
>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>>>  - Correct auto suspend timer value for suspend/resume issue
>>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>>>  - Fix KE due to no sen-inf sub-device
>>> ---
>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>
>> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
>> capture, which for metadata, and what is being used to deal with requests.
>>
>> It would make it easier to review imho.
>>
> 
> For file structure design, it was reviewed in the previous patch
> serials.
> e.g.
> https://patchwork.kernel.org/patch/10938137/
> If you think it is better, I will modify it.

Right, I saw a suggestion to merge two files there.

I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
So it reflects a bit the topology.
But it is also up to you to see if it improves organization or not, it is just a suggestion.

> 
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>
>> It would be nice to chose beween mtk_cam or mtk-isp for naming functions, files and configs, and keep consistency.
>>
>> Or maybe something like:
>>
>> mtkisp_p1_core.c (with probe, who creates all the media entities, deals with fwnodes, etc)
>> mtkisp_p1_capture.c
>> mtkisp_p1_meta.c
>> mtkisp_p1_isp.c
>> mtkisp_p1_hw.c (or maybe split this between the other files)
>> mtkisp_p1_request.c
>> mtkisp_p1_common.c (?)
>>
>> or s/mtkisp_p1/mtk_cam/
>>
>> what do you think?
>>
> 
> Ok, I will revise our naming issue for consistency reason.
> 
>>>  9 files changed, 3377 insertions(+)
>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>
>>> diff --git a/drivers/media/platform/mtk-isp/Kconfig b/drivers/media/platform/mtk-isp/Kconfig
>>> new file mode 100644
>>> index 000000000000..f86e1b59ad1e
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/Kconfig
>>> @@ -0,0 +1,20 @@
>>> +config VIDEO_MEDIATEK_ISP_PASS1
>>> +	tristate "Mediatek ISP Pass 1 driver"
>>> +	depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
>>
>> I think you need OF as well
>>
>> depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF
>>
>>> +	depends on ARCH_MEDIATEK
>>
>> depends on ARCH_MEDIATEK || COMPILE_TEST
>>
> 
> Ok, we will fix this in next patch.
> 
>>> +	select V4L2_FWNODE
>>> +	select VIDEOBUF2_VMALLOC
>>> +	select VIDEOBUF2_DMA_CONTIG
>>> +	select MTK_SCP
>>> +	default n
>>> +	help
>>> +		Pass 1 driver controls 3A (auto-focus, exposure,
>>> +		and white balance) with tuning feature and outputs
>>> +		the captured image buffers in Mediatek's camera system.
>>> +
>>> +		Choose Y if you want to use Mediatek SoCs to create image
>>> +		captured application such as video recording and still image
>>> +		capturing.
>>
>> I would re-word this a bit, since people can use a captured application (and not create one) :)
>>
> 
> Ok, I will re-word as "if you want to use image captured application
> based on Mediatek SoCs for video recording and still image capturing
> functions"
> 
>>> +
>>> +		To compile this driver as a module, choose M here; the module
>>> +		will be called mtk-cam-isp.
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/Makefile b/drivers/media/platform/mtk-isp/isp_50/Makefile
>>> new file mode 100644
>>> index 000000000000..ce79d283b209
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/Makefile
>>> @@ -0,0 +1,3 @@
>>> +# SPDX-License-Identifier: GPL-2.0
>>> +
>>> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += cam/
>>> \ No newline at end of file
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/Makefile b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>> new file mode 100644
>>> index 000000000000..53b54d3c26a0
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>> @@ -0,0 +1,6 @@
>>> +# SPDX-License-Identifier: GPL-2.0
>>> +
>>> +mtk-cam-isp-objs += mtk_cam.o
>>> +mtk-cam-isp-objs += mtk_cam-hw.o
>>> +
>>> +obj-$(CONFIG_VIDEO_MEDIATEK_ISP_PASS1) += mtk-cam-isp.o
>>> \ No newline at end of file
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>> new file mode 100644
>>> index 000000000000..4065d0d29b7f
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>> @@ -0,0 +1,636 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +//
>>> +// Copyright (c) 2019 MediaTek Inc.
>>> +
>>> +#include <linux/atomic.h>
>>> +#include <linux/clk.h>
>>> +#include <linux/interrupt.h>
>>> +#include <linux/iopoll.h>
>>> +#include <linux/of_platform.h>
>>> +#include <linux/of_irq.h>
>>> +#include <linux/module.h>
>>> +#include <linux/remoteproc/mtk_scp.h>
>>> +#include <linux/pm_runtime.h>
>>> +#include <linux/remoteproc.h>
>>> +#include <linux/sched.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/types.h>
>>> +#include <linux/videodev2.h>
>>> +#include <linux/vmalloc.h>
>>> +
>>> +#include <media/v4l2-event.h>
>>
>> Please sort headers alphabetically.
>>
> 
> Will fix in next patch.
> 
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-hw.h"
>>> +#include "mtk_cam-regs.h"
>>> +
>>> +#define MTK_ISP_COMPOSER_MEM_SIZE		0x200000
>>> +#define MTK_ISP_CQ_BUFFER_COUNT			3
>>> +#define MTK_ISP_CQ_ADDRESS_OFFSET		0x640
>>> +
>>> +/*
>>> + *
>>> + * MTK Camera ISP P1 HW supports 3 ISP HW (CAM A/B/C).
>>> + * The T-put capability of CAM B is the maximum (max line buffer: 5376 pixels)
>>> + * For CAM A/C, it only supports max line buffer with 3328 pixels.
>>> + * In current driver, only supports CAM B.
>>> + *
>>> + */
>>> +#define MTK_ISP_CAM_ID_B			3
>>> +#define MTK_ISP_AUTOSUSPEND_DELAY_MS		66
>>> +#define MTK_ISP_IPI_SEND_TIMEOUT		1000
>>> +#define MTK_ISP_STOP_HW_TIMEOUT			(33 * USEC_PER_MSEC)
>>> +
>>> +static void isp_tx_frame_worker(struct work_struct *work)
>>
>> I suggest prefixing all the function and macros with mtk_isp_, it is easier to know they are not
>> an external function.
>>
> 
> Fix in next patch.
> 
>>> +{
>>> +	struct mtk_cam_dev_request *req =
>>> +		container_of(work, struct mtk_cam_dev_request, frame_work);
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(req->req.mdev, struct mtk_cam_dev, media_dev);
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_FRAME, &req->frame_params,
>>> +		     sizeof(req->frame_params), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +static void isp_composer_handler(void *data, unsigned int len, void *priv)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)priv;
>>> +	struct device *dev = p1_dev->dev;
>>> +	struct mtk_isp_scp_p1_cmd *ipi_msg;
>>> +
>>> +	ipi_msg = (struct mtk_isp_scp_p1_cmd *)data;
>>> +
>>> +	if (len < offsetofend(struct mtk_isp_scp_p1_cmd, ack_info)) {
>>> +		dev_err(dev, "wrong IPI len:%d\n", len);
>>> +		return;
>>> +	}
>>> +
>>> +	if (ipi_msg->cmd_id != ISP_CMD_ACK ||
>>> +	    ipi_msg->ack_info.cmd_id != ISP_CMD_FRAME_ACK)
>>> +		return;
>>> +
>>> +	p1_dev->composed_frame_seq_no = ipi_msg->ack_info.frame_seq_no;
>>> +	dev_dbg(dev, "ack frame_num:%d\n", p1_dev->composed_frame_seq_no);
>>> +}
>>> +
>>> +static int isp_composer_init(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct device *dev = p1_dev->dev;
>>> +	int ret;
>>> +
>>> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_CMD,
>>> +			       isp_composer_handler, p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register IPI cmd\n");
>>> +		return ret;
>>> +	}
>>> +	ret = scp_ipi_register(p1_dev->scp, SCP_IPI_ISP_FRAME,
>>> +			       isp_composer_handler, p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register IPI frame\n");
>>> +		goto unreg_ipi_cmd;
>>> +	}
>>> +
>>> +	p1_dev->composer_wq =
>>> +		alloc_ordered_workqueue(dev_name(p1_dev->dev),
>>> +					__WQ_LEGACY | WQ_MEM_RECLAIM |
>>> +					WQ_FREEZABLE);
>>> +	if (!p1_dev->composer_wq) {
>>> +		dev_err(dev, "failed to alloc composer workqueue\n");
>>> +		goto unreg_ipi_frame;
>>> +	}
>>> +
>>> +	return 0;
>>> +
>>> +unreg_ipi_frame:
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
>>> +unreg_ipi_cmd:
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void isp_composer_uninit(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	destroy_workqueue(p1_dev->composer_wq);
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_CMD);
>>> +	scp_ipi_unregister(p1_dev->scp, SCP_IPI_ISP_FRAME);
>>> +}
>>> +
>>> +static void isp_composer_hw_init(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_INIT;
>>> +	composer_tx_cmd.init_param.hw_module = MTK_ISP_CAM_ID_B;
>>> +
>>> +	/*
>>> +	 * Passed coherent reserved memory info. for SCP firmware usage.
>>> +	 * This buffer is used for SCP's ISP composer to compose.
>>> +	 * The size of is fixed to 0x200000 for the requirement of composer.
>>> +	 */
>>> +	composer_tx_cmd.init_param.cq_addr.iova = p1_dev->composer_iova;
>>> +	composer_tx_cmd.init_param.cq_addr.scp_addr = p1_dev->composer_scp_addr;
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +static void isp_composer_hw_deinit(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_DEINIT;
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +
>>> +	isp_composer_uninit(p1_dev);
>>
>> I think you can copy the 3 lines of this isp_composer_uninit() function here, since
>> this seems the only place it is being used, and having a deinit and uninit function is
>> a bit confusing.
>>
> 
> Fix in next patch.
> 
>>> +}
>>> +
>>> +void mtk_isp_hw_config(struct mtk_cam_dev *cam,
>>> +		       struct p1_config_param *config_param)
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_CONFIG;
>>> +	memcpy(&composer_tx_cmd.config_param, config_param,
>>> +	       sizeof(*config_param));
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +void mtk_isp_stream(struct mtk_cam_dev *cam, int on)
>>
>> I prefer not having a int parameter, this is easier to read:
>>
>> mtk_isp_stream_on(cam);
>> mtk_isp_stream_off(cam);
>>
>> or
>>
>> mtk_isp_stream(cam, MTK_ISP_STREAM_ON);
>> mtk_isp_stream(cam, MTK_ISP_STREAM_OFF);
>>
>> instead of:
>>
>> mtk_isp_stream(cam, 1);
>> mtk_isp_stream(cam, 0);
>>
>> You can add wrappers to this function, and leave this one (that receives the boolean parameter) internal.
>>
> 
> Ok, I will choose the method 2.
> 
>>> +{
>>> +	struct mtk_isp_scp_p1_cmd composer_tx_cmd;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	memset(&composer_tx_cmd, 0, sizeof(composer_tx_cmd));
>>> +	composer_tx_cmd.cmd_id = ISP_CMD_STREAM;
>>> +	composer_tx_cmd.is_stream_on = on;
>>
>> s/is_stream_on/is_streaming
>>
> 
> Fix in next patch.
> 
>>> +
>>> +	scp_ipi_send(p1_dev->scp, SCP_IPI_ISP_CMD, &composer_tx_cmd,
>>> +		     sizeof(composer_tx_cmd), MTK_ISP_IPI_SEND_TIMEOUT);
>>> +}
>>> +
>>> +int mtk_isp_hw_init(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	int ret;
>>> +
>>> +	ret = rproc_boot(p1_dev->rproc_handle);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to rproc_boot\n");
>>
>> It would be nice to improve this error message for users, how about:
>>
>> dev_err(dev, "Initialization of remote processor %s failed", p1_dev->rproc_handle);
>>
>> Or maybe even remove this message, since rproc_boot() already have several error messages.
>>
> 
> Ok, we will remove the error message.
> 
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = isp_composer_init(p1_dev);
>>> +	if (ret)
>>
>> should rproc_shutdown() be called here?
>>
> 
> Yes, we will fix it.
> 
>>> +		return ret;
>>> +
>>> +	pm_runtime_get_sync(dev);
>>
>> You should check return value here.
>>
> 
> Fix in next patch.
> 
>>> +	isp_composer_hw_init(p1_dev);
>>> +
>>> +	p1_dev->enqueued_frame_seq_no = 0;
>>> +	p1_dev->dequeued_frame_seq_no = 0;
>>> +	p1_dev->composed_frame_seq_no = 0;
>>> +	p1_dev->sof_count = 0;
>>> +
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	isp_composer_hw_deinit(p1_dev);
>>> +	pm_runtime_mark_last_busy(dev);
>>> +	pm_runtime_put_autosuspend(dev);
>>> +	rproc_shutdown(p1_dev->rproc_handle);
>>> +
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>>> +			 struct mtk_cam_dev_request *req)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>> +
>>> +	/* Accumulated frame sequence number */
>>> +	req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
>>> +
>>> +	INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>>> +	queue_work(p1_dev->composer_wq, &req->frame_work);
>>> +	dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>>> +		req->req.debug_str, req->frame_params.frame_seq_no,
>>> +		cam->running_job_count);
>>> +}
>>> +
>>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>>> +			       unsigned int dequeued_frame_seq_no)
>>> +{
>>> +	dma_addr_t base_addr = p1_dev->composer_iova;
>>> +	struct device *dev = p1_dev->dev;
>>> +	struct mtk_cam_dev_request *req;
>>> +	int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
>>> +	unsigned int addr_offset;
>>> +
>>> +	/* Send V4L2_EVENT_FRAME_SYNC event */
>>> +	mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>> +
>>> +	p1_dev->sof_count += 1;
>>> +	/* Save frame information */
>>> +	p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
>>> +
>>> +	req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>> +	if (req)
>>> +		req->timestamp = ktime_get_boottime_ns();
>>> +
>>> +	/* Update CQ base address if needed */
>>> +	if (composed_frame_seq_no <= dequeued_frame_seq_no) {
>>> +		dev_dbg(dev,
>>> +			"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
>>> +			composed_frame_seq_no, dequeued_frame_seq_no);
>>> +		return;
>>> +	}
>>> +	addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>>> +		(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>>> +	writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>>> +	dev_dbg(dev,
>>> +		"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
>>> +		composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
>>> +}
>>> +
>>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	u32 val;
>>> +
>>> +	dev_err(p1_dev->dev,
>>> +		"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
>>> +		readl(p1_dev->regs + REG_IMGO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_RRZO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_AAO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_AFO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_LMVO_ERR_STAT));
>>> +	dev_err(p1_dev->dev,
>>> +		"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
>>> +		readl(p1_dev->regs + REG_LCSO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_PSO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_FLKO_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_BPCI_ERR_STAT),
>>> +		readl(p1_dev->regs + REG_LSCI_ERR_STAT));
>>
>> I think if would be better to transfor those into dev_dbg and add a counter
>> in debugfs.
>>
> 
> These error messages are important for debugging.
> I suggest to keep in dev_err.

I mean, these messages are usefull for debug (as you mentioned yourself), but for an
end user not so much, since end users won't know the meaning of those values.

For end users a "dma failure" message would be enough, then advanced users can enable
debug messages to see more.

> 
> Moreover, could you give more information about debug counter?
> I don't get your point.
> Do you suggest to accumulate the total count of DMA errors?


Yes, you could have a debugfs entry with error counters like:

cat /debugfs/mtk_isp/dma_err
8

So it is easier if this error happens very frequent or not.
In the rkisp1 case we added:

/debugfs/rkisp1/data_loss
/debugfs/rkisp1/pic_size_error
/debugfs/rkisp1/mipi_error
/debugfs/rkisp1/stats_error
/debugfs/rkisp1/mp_stop_timeout
/debugfs/rkisp1/sp_stop_timeout
/debugfs/rkisp1/mp_frame_drop
/debugfs/rkisp1/sp_frame_drop

Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.

But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).

> 
>>> +
>>> +	/* Disable DMA error mask to avoid too much error log */
>>> +	val = readl(p1_dev->regs + REG_CTL_RAW_INT_EN);
>>> +	writel((val & (~DMA_ERR_INT_EN)), p1_dev->regs + REG_CTL_RAW_INT_EN);
>>> +	dev_dbg(p1_dev->dev, "disable DMA error mask:0x%x\n", val);
>>> +}
>>> +
>>> +static irqreturn_t isp_irq_cam(int irq, void *data)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = (struct mtk_isp_p1_device *)data;
>>> +	struct device *dev = p1_dev->dev;
>>> +	unsigned int dequeued_frame_seq_no;
>>> +	unsigned int irq_status, err_status, dma_status;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&p1_dev->spinlock_irq, flags);
>>> +	irq_status = readl(p1_dev->regs + REG_CTL_RAW_INT_STAT);
>>> +	err_status = irq_status & INT_ST_MASK_CAM_ERR;
>>> +	dma_status = readl(p1_dev->regs + REG_CTL_RAW_INT2_STAT);
>>> +	dequeued_frame_seq_no = readl(p1_dev->regs + REG_FRAME_SEQ_NUM);
>>> +	spin_unlock_irqrestore(&p1_dev->spinlock_irq, flags);
>>> +
>>> +	/*
>>> +	 * In normal case, the next SOF ISR should come after HW PASS1 DONE ISR.
>>> +	 * If these two ISRs come together, print warning msg to hint.
>>> +	 */
>>> +	if ((irq_status & SOF_INT_ST) && (irq_status & HW_PASS1_DON_ST))
>>> +		dev_dbg(dev, "sof_done block cnt:%d\n", p1_dev->sof_count);
>>> +
>>> +	/* De-queue frame */
>>> +	if (irq_status & SW_PASS1_DON_ST) {
>>
>> I suppose this means "done streaming"?
>>
> 
> Yes, it means the frame buffer is outputed done.
> 
>>> +		mtk_cam_dev_dequeue_req_frame(&p1_dev->cam_dev,
>>> +					      p1_dev->dequeued_frame_seq_no);
>>> +		mtk_cam_dev_req_try_queue(&p1_dev->cam_dev);
>>> +	}
>>> +
>>> +	/* Save frame info. & update CQ address for frame HW en-queue */
>>> +	if (irq_status & SOF_INT_ST)
>>> +		isp_irq_handle_sof(p1_dev, dequeued_frame_seq_no);
>>> +
>>> +	/* Check ISP error status */
>>> +	if (err_status) {
>>> +		dev_err(dev, "int_err:0x%x 0x%x\n", irq_status, err_status);
>>> +		/* Show DMA errors in detail */
>>> +		if (err_status & DMA_ERR_ST)
>>> +			isp_irq_handle_dma_err(p1_dev);
>>> +	}
>>> +
>>> +	dev_dbg(dev, "SOF:%d irq:0x%x, dma:0x%x, frame_num:%d\n",
>>> +		p1_dev->sof_count, irq_status, dma_status,
>>> +		dequeued_frame_seq_no);
>>> +
>>> +	return IRQ_HANDLED;
>>> +}
>>> +
>>> +static int isp_setup_scp_rproc(struct mtk_isp_p1_device *p1_dev,
>>> +			       struct platform_device *pdev)
>>> +{
>>> +	struct device *dev = p1_dev->dev;
>>> +	dma_addr_t addr;
>>> +	void *ptr;
>>
>> Maybe "composer_buffer" would be a better name.
>>
>> But is this variable required at all? Can't it be allocated directly to p1_dev->composer_virt_addr ?
>>
> 
> Ok, I will use p1_dev->composer_virt_addr directly.
> 
>>> +	int ret;
>>> +
>>> +	p1_dev->scp = scp_get(pdev);
>>> +	if (!p1_dev->scp) {
>>> +		dev_err(dev, "failed to get scp device\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	p1_dev->rproc_handle = scp_get_rproc(p1_dev->scp);
>>> +	dev_dbg(dev, "p1 rproc_phandle: 0x%pK\n", p1_dev->rproc_handle);
>>> +	p1_dev->cam_dev.smem_dev = scp_get_device(p1_dev->scp);
>>
>> I would rename smem_dev to scp_dev, this helps making it clear when allocating dma buffers
>> which mapping we are refering to.
>>
> 
> Fix in next patch.
> 
>>> +
>>> +	/*
>>> +	 * Allocate coherent reserved memory for SCP firmware usage.
>>> +	 * The size of SCP composer's memory is fixed to 0x200000
>>> +	 * for the requirement of firmware.
>>> +	 */
>>> +	ptr = dma_alloc_coherent(p1_dev->cam_dev.smem_dev,
>>> +				 MTK_ISP_COMPOSER_MEM_SIZE, &addr, GFP_KERNEL);
>>> +	if (!ptr) {
>>> +		ret = -ENOMEM;
>>> +		goto fail_put_scp;
>>> +	}
>>> +
>>> +	p1_dev->composer_scp_addr = addr;
>>> +	p1_dev->composer_virt_addr = ptr;
>>> +	dev_dbg(dev, "scp addr:%pad va:%pK\n", &addr, ptr);
>>> +
>>> +	/*
>>> +	 * This reserved memory is also be used by ISP P1 HW.
>>> +	 * Need to get iova address for ISP P1 DMA.
>>> +	 */
>>> +	addr = dma_map_resource(dev, addr, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +				DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
>>> +	if (dma_mapping_error(dev, addr)) {
>>> +		dev_err(dev, "failed to map scp iova\n");
>>> +		ret = -ENOMEM;
>>> +		goto fail_free_mem;
>>> +	}
>>> +	p1_dev->composer_iova = addr;
>>
>> why not rename this to composer_isp_addr ?
>> Since, afaik, composer_scp_addr is also iova.
>>
>> At least my concept of iova (IO virtual address), are an address behind an IOMMU (or bus address to be given to a device).
>>
> 
> Ok, we will rename composer_iova to composer_isp_addr.
> Basically, scp_addr is reserved physical address and it is not behind an
> IOMMU.
> 
>>> +	dev_dbg(dev, "scp iova addr:%pad\n", &addr);
>>> +
>>> +	return 0;
>>> +
>>> +fail_free_mem:
>>> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +			  p1_dev->composer_virt_addr,
>>> +			  p1_dev->composer_scp_addr);
>>> +	p1_dev->composer_scp_addr = 0;
>>> +fail_put_scp:
>>> +	scp_put(p1_dev->scp);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void isp_teardown_scp_rproc(struct mtk_isp_p1_device *p1_dev)
>>> +{
>>> +	dma_free_coherent(p1_dev->cam_dev.smem_dev, MTK_ISP_COMPOSER_MEM_SIZE,
>>> +			  p1_dev->composer_virt_addr,
>>> +			  p1_dev->composer_scp_addr);
>>> +	p1_dev->composer_scp_addr = 0;
>>> +	scp_put(p1_dev->scp);
>>> +}
>>> +
>>> +static int mtk_isp_pm_suspend(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	u32 val;
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "- %s\n", __func__);
>>> +
>>> +	if (pm_runtime_suspended(dev))
>>> +		return 0;
>>> +
>>> +	/* Disable ISP's view finder and wait for TG idle if possible */
>>> +	dev_dbg(dev, "cam suspend, disable VF\n");
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val & (~TG_VF_CON_VFDATA_EN), p1_dev->regs + REG_TG_VF_CON);
>>> +	readl_poll_timeout_atomic(p1_dev->regs + REG_TG_INTER_ST, val,
>>> +				  (val & TG_CS_MASK) == TG_IDLE_ST,
>>> +				  USEC_PER_MSEC, MTK_ISP_STOP_HW_TIMEOUT);
>>> +
>>> +	/* Disable CMOS */
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val & (~TG_SEN_MODE_CMOS_EN), p1_dev->regs + REG_TG_SEN_MODE);
>>> +
>>> +	/* Force ISP HW to idle */
>>> +	ret = pm_runtime_force_suspend(dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to force suspend:%d\n", ret);
>>> +		goto reenable_hw;
>>> +	}
>>> +
>>> +	return 0;
>>> +
>>> +reenable_hw:
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_isp_pm_resume(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	u32 val;
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "- %s\n", __func__);
>>> +
>>> +	if (pm_runtime_suspended(dev))
>>> +		return 0;
>>> +
>>> +	/* Force ISP HW to resume */
>>> +	ret = pm_runtime_force_resume(dev);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	/* Enable CMOS */
>>> +	dev_dbg(dev, "cam resume, enable CMOS/VF\n");
>>> +	val = readl(p1_dev->regs + REG_TG_SEN_MODE);
>>> +	writel(val | TG_SEN_MODE_CMOS_EN, p1_dev->regs + REG_TG_SEN_MODE);
>>> +
>>> +	/* Enable VF */
>>> +	val = readl(p1_dev->regs + REG_TG_VF_CON);
>>> +	writel(val | TG_VF_CON_VFDATA_EN, p1_dev->regs + REG_TG_VF_CON);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_runtime_suspend(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	dev_dbg(dev, "%s:disable clock\n", __func__);
>>> +	clk_bulk_disable_unprepare(p1_dev->num_clks, p1_dev->clks);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_runtime_resume(struct device *dev)
>>> +{
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +	int ret;
>>> +
>>> +	dev_dbg(dev, "%s:enable clock\n", __func__);
>>> +	ret = clk_bulk_prepare_enable(p1_dev->num_clks, p1_dev->clks);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to enable clock:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_probe(struct platform_device *pdev)
>>> +{
>>> +	/* List of clocks required by isp cam */
>>> +	static const char * const clk_names[] = {
>>> +		"camsys_cam_cgpdn", "camsys_camtg_cgpdn"
>>> +	};
>>> +	struct mtk_isp_p1_device *p1_dev;
>>> +	struct device *dev = &pdev->dev;
>>> +	struct resource *res;
>>> +	int irq, ret, i;
>>> +
>>> +	p1_dev = devm_kzalloc(dev, sizeof(*p1_dev), GFP_KERNEL);
>>> +	if (!p1_dev)
>>> +		return -ENOMEM;
>>> +
>>> +	p1_dev->dev = dev;
>>> +	dev_set_drvdata(dev, p1_dev);
>>> +
>>> +	/*
>>> +	 * Now only support single CAM with CAM B.
>>> +	 * Get CAM B register base with CAM B index.
>>> +	 * Support multiple CAMs in future.
>>> +	 */
>>> +	res = platform_get_resource(pdev, IORESOURCE_MEM, MTK_ISP_CAM_ID_B);
>>> +	p1_dev->regs = devm_ioremap_resource(dev, res);
>>> +	if (IS_ERR(p1_dev->regs)) {
>>> +		dev_err(dev, "failed to map reister base\n");
>>
>> s/reister/register
>>
> 
> Fix in next patch.
> 
>>> +		return PTR_ERR(p1_dev->regs);
>>> +	}
>>> +	dev_dbg(dev, "cam, map_addr=0x%pK\n", p1_dev->regs);
>>> +
>>> +	/*
>>> +	 * The cam_sys unit only supports reg., but has no IRQ support.
>>> +	 * The reg. & IRQ index is shifted with 1 for CAM B in DTS.
>>> +	 */
>>> +	irq = platform_get_irq(pdev, MTK_ISP_CAM_ID_B - 1);
>>> +	if (!irq) {
>>> +		dev_err(dev, "failed to get irq\n");
>>> +		return -ENODEV;
>>> +	}
>>> +	ret = devm_request_irq(dev, irq, isp_irq_cam, 0, dev_name(dev),
>>> +			       p1_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to request irq=%d\n", irq);
>>> +		return ret;
>>> +	}
>>> +	dev_dbg(dev, "registered irq=%d\n", irq);
>>> +	spin_lock_init(&p1_dev->spinlock_irq);
>>> +
>>> +	p1_dev->num_clks = ARRAY_SIZE(clk_names);
>>> +	p1_dev->clks = devm_kcalloc(dev, p1_dev->num_clks,
>>> +				    sizeof(*p1_dev->clks), GFP_KERNEL);
>>> +	if (!p1_dev->clks)
>>> +		return -ENOMEM;
>>> +
>>> +	for (i = 0; i < p1_dev->num_clks; ++i)
>>> +		p1_dev->clks[i].id = clk_names[i];
>>> +
>>> +	ret = devm_clk_bulk_get(dev, p1_dev->num_clks, p1_dev->clks);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to get isp cam clock:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = isp_setup_scp_rproc(p1_dev, pdev);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	pm_runtime_set_autosuspend_delay(dev, MTK_ISP_AUTOSUSPEND_DELAY_MS);
>>> +	pm_runtime_use_autosuspend(dev);
>>> +	pm_runtime_enable(dev);
>>> +
>>> +	/* Initialize the v4l2 common part */
>>> +	ret = mtk_cam_dev_init(pdev, &p1_dev->cam_dev);
>>> +	if (ret) {
>>> +		isp_teardown_scp_rproc(p1_dev);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_isp_remove(struct platform_device *pdev)
>>> +{
>>> +	struct device *dev = &pdev->dev;
>>> +	struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>> +
>>> +	mtk_cam_dev_cleanup(&p1_dev->cam_dev);
>>> +	pm_runtime_dont_use_autosuspend(dev);
>>> +	pm_runtime_disable(dev);
>>> +	dma_unmap_page_attrs(dev, p1_dev->composer_iova,
>>> +			     MTK_ISP_COMPOSER_MEM_SIZE, DMA_TO_DEVICE,
>>> +			     DMA_ATTR_SKIP_CPU_SYNC);
>>> +	isp_teardown_scp_rproc(p1_dev);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static const struct dev_pm_ops mtk_isp_pm_ops = {
>>> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_isp_pm_suspend, mtk_isp_pm_resume)
>>> +	SET_RUNTIME_PM_OPS(mtk_isp_runtime_suspend, mtk_isp_runtime_resume,
>>> +			   NULL)
>>> +};
>>> +
>>> +static const struct of_device_id mtk_isp_of_ids[] = {
>>> +	{.compatible = "mediatek,mt8183-camisp",},
>>> +	{}
>>> +};
>>> +MODULE_DEVICE_TABLE(of, mtk_isp_of_ids);
>>> +
>>> +static struct platform_driver mtk_isp_driver = {
>>> +	.probe   = mtk_isp_probe,
>>> +	.remove  = mtk_isp_remove,
>>> +	.driver  = {
>>> +		.name  = "mtk-cam-p1",
>>> +		.of_match_table = of_match_ptr(mtk_isp_of_ids),
>>> +		.pm     = &mtk_isp_pm_ops,
>>> +	}
>>> +};
>>> +
>>> +module_platform_driver(mtk_isp_driver);
>>> +
>>> +MODULE_DESCRIPTION("Mediatek ISP P1 driver");
>>> +MODULE_LICENSE("GPL v2");
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>> new file mode 100644
>>> index 000000000000..837662f92a5e
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>
>> This header file is really short, why not merge it with mtk_cam.h (that is small too) and call it mtk_isp_common.h or mtk_cam_common?
>>
> 
> Ok, revise in next patch.
> 
>>> @@ -0,0 +1,64 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_HW_H__
>>> +#define __MTK_CAM_HW_H__
>>> +
>>> +#include <linux/types.h>
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-ipi.h"
>>> +
>>> +/*
>>> + * struct mtk_isp_p1_device - the Mediatek ISP P1 device information
>>> + *
>>> + * @dev: Pointer to device.
>>> + * @scp_pdev: Pointer to SCP platform device.
>>> + * @rproc_handle: Pointer to new remoteproc instance.
>>> + * @cam_dev: Embedded struct cam_dev
>>> + * @regs: Camera ISP HW base register address
>>> + * @num_clks: The number of driver's clocks
>>> + * @clks: The clock data array
>>> + * @spinlock_irq: Used to protect register read/write data
>>> + * @enqueued_frame_seq_no: Frame sequence number of enqueued frame
>>> + * @dequeued_frame_seq_no: Frame sequence number of dequeued frame
>>> + * @composed_frame_seq_no: Frame sequence number of composed frame
>>> + * @timestamp: Frame timestamp in ns
>>> + * @sof_count: SOF counter
>>> + * @composer_wq: The work queue for frame request composing
>>> + * @composer_scp_addr: SCP address of ISP composer memory
>>> + * @composer_iova: DMA address of ISP composer memory
>>> + * @virt_addr: Virtual address of ISP composer memory
>>> + *
>>> + */
>>> +struct mtk_isp_p1_device {
>>> +	struct device *dev;
>>> +	struct mtk_scp *scp;
>>> +	struct rproc *rproc_handle;
>>> +	struct mtk_cam_dev cam_dev;
>>> +	void __iomem *regs;
>>> +	unsigned int num_clks;
>>> +	struct clk_bulk_data *clks;
>>> +	/* Used to protect register read/write data */
>>> +	spinlock_t spinlock_irq;
>>> +	unsigned int enqueued_frame_seq_no;
>>> +	unsigned int dequeued_frame_seq_no;
>>> +	unsigned int composed_frame_seq_no;
>>> +	u8 sof_count;
>>> +	struct workqueue_struct *composer_wq;
>>> +	dma_addr_t composer_scp_addr;
>>> +	dma_addr_t composer_iova;
>>> +	void *composer_virt_addr;
>>> +};
>>> +
>>> +int mtk_isp_hw_init(struct mtk_cam_dev *cam_dev);
>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam_dev);
>>> +void mtk_isp_hw_config(struct mtk_cam_dev *cam_dev,
>>> +		       struct p1_config_param *config_param);
>>> +void mtk_isp_stream(struct mtk_cam_dev *cam_dev, int on);
>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam_dev,
>>> +			 struct mtk_cam_dev_request *req);
>>
>> It would be nice to have docs for these too.
>>
> 
> Ok, add in next patch.
> 
>>> +
>>> +#endif /* __MTK_CAM_HW_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>> new file mode 100644
>>> index 000000000000..981b634dd91f
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>
>> I'm skipping this file, since, if I understand correctly, this is not ready for review right?
>>
> 
> I think this file is ready for review.
> 
> 
>>> @@ -0,0 +1,222 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_IPI_H__
>>> +#define __MTK_CAM_IPI_H__
>>> +
>>> +#include <linux/types.h>
>>> +
>>> +/*
>>> + * struct img_size - Image size information.
>>> + *
>>> + * @w: Image width, the unit is pixel
>>> + * @h: Image height, the unit is pixel
>>> + * @xsize: Bytes per line based on width.
>>> + * @stride: Bytes per line when changing line.
>>> + *          Stride is based on xsize + HW constrain(byte align).
>>> + *
>>> + */
>>> +struct img_size {
>>> +	u32 w;
>>> +	u32 h;
>>> +	u32 xsize;
>>> +	u32 stride;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_img_crop - image corp information
>>> + *
>>> + * @left: The left of crop area.
>>> + * @top: The top of crop area.
>>> + * @width: The width of crop area.
>>> + * @height: The height of crop area.
>>> + *
>>> + */
>>> +struct p1_img_crop {
>>> +	u32 left;
>>> +	u32 top;
>>> +	u32 width;
>>> +	u32 height;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct dma_buffer - DMA buffer address information
>>> + *
>>> + * @iova: DMA address for ISP DMA device
>>> + * @scp_addr: SCP address for external co-process unit
>>> + *
>>> + */
>>> +struct dma_buffer {
>>> +	u32 iova;
>>
>> I would rename this to isp_addr, since scp_addr is also iova (at least this is the way I understand).
>>
> 
> Ok, revise in next patch.
> 
>>> +	u32 scp_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_img_output - ISP P1 image output information
>>> + *
>>> + * @buffer: DMA buffer address of image.
>>> + * @size: The image size configuration.
>>> + * @crop: The crop configuration.
>>> + * @pixel_bits: The bits per image pixel.
>>> + * @img_fmt: The image format.
>>> + *
>>> + */
>>> +struct p1_img_output {
>>> +	struct dma_buffer buffer;
>>> +	struct img_size size;
>>> +	struct p1_img_crop crop;
>>> +	u8 pixel_bits;
>>> +	u32 img_fmt;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_in_param - Image input parameters structure.
>>> + *                       Normally, it comes from sensor information.
>>> + *
>>> + * @continuous: Indicate the sensor mode. Continuous or single shot.
>>> + * @subsample: Indicate to enables SOF subsample or not.
>>> + * @pixel_mode: Describe 1/2/4 pixels per clock cycle.
>>> + * @data_pattern: Describe input data pattern.
>>> + * @raw_pixel_id: Bayer sequence.
>>> + * @tg_fps: The fps rate of TG (time generator).
>>> + * @img_fmt: The image format of input source.
>>> + * @p1_img_crop: The crop configuration of input source.
>>> + *
>>> + */
>>> +struct cfg_in_param {
>>> +	u8 continuous;
>>> +	u8 subsample;
>>> +	u8 pixel_mode;
>>> +	u8 data_pattern;
>>> +	u8 raw_pixel_id;
>>> +	u16 tg_fps;
>>> +	u32 img_fmt;
>>> +	struct p1_img_crop crop;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_main_out_param - The image output parameters of main stream.
>>> + *
>>> + * @bypass: Indicate this device is enabled or disabled or not.
>>> + * @pure_raw: Indicate the image path control.
>>> + *            True: pure raw
>>> + *            False: processing raw
>>> + * @pure_raw_pack: Indicate the image is packed or not.
>>> + *                 True: packed mode
>>> + *                 False: unpacked mode
>>> + * @p1_img_output: The output image information.
>>> + *
>>> + */
>>> +struct cfg_main_out_param {
>>> +	u8 bypass;
>>> +	u8 pure_raw;
>>> +	u8 pure_raw_pack;
>>> +	struct p1_img_output output;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct cfg_resize_out_param - The image output parameters of
>>> + *                               packed out stream.
>>> + *
>>> + * @bypass: Indicate this device is enabled or disabled or not.
>>> + * @p1_img_output: The output image information.
>>> + *
>>> + */
>>> +struct cfg_resize_out_param {
>>> +	u8 bypass;
>>> +	struct p1_img_output output;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct p1_config_param - ISP P1 configuration parameters.
>>> + *
>>> + * @cfg_in_param: The Image input parameters.
>>> + * @cfg_main_param: The main output image parameters.
>>> + * @cfg_resize_out_param: The packed output image parameters.
>>> + * @enabled_dmas: The enabled DMA port information.
>>> + *
>>> + */
>>> +struct p1_config_param {
>>> +	struct cfg_in_param cfg_in_param;
>>> +	struct cfg_main_out_param cfg_main_param;
>>> +	struct cfg_resize_out_param cfg_resize_param;
>>> +	u32 enabled_dmas;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct P1_meta_frame - ISP P1 meta frame information.
>>> + *
>>> + * @enabled_dma: The enabled DMA port information.
>>> + * @vb_index: The VB2 index of meta buffer.
>>> + * @meta_addr: DMA buffer address of meta buffer.
>>> + *
>>> + */
>>> +struct P1_meta_frame {
>>> +	u32 enabled_dma;
>>> +	u32 vb_index;
>>> +	struct dma_buffer meta_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct isp_init_info - ISP P1 composer init information.
>>> + *
>>> + * @hw_module: The ISP Camera HW module ID.
>>> + * @cq_addr: The DMA address of composer memory.
>>> + *
>>> + */
>>> +struct isp_init_info {
>>> +	u8 hw_module;
>>> +	struct dma_buffer cq_addr;
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct isp_ack_info - ISP P1 IPI command ack information.
>>> + *
>>> + * @cmd_id: The IPI command ID is acked.
>>> + * @frame_seq_no: The IPI frame sequence number is acked.
>>> + *
>>> + */
>>> +struct isp_ack_info {
>>> +	u8 cmd_id;
>>> +	u32 frame_seq_no;
>>> +} __packed;
>>> +
>>> +/*
>>> + * The IPI command enumeration.
>>> + */
>>> +enum mtk_isp_scp_cmds {
>>> +	ISP_CMD_INIT,
>>> +	ISP_CMD_CONFIG,
>>> +	ISP_CMD_STREAM,
>>> +	ISP_CMD_DEINIT,
>>> +	ISP_CMD_ACK,
>>> +	ISP_CMD_FRAME_ACK,
>>> +	ISP_CMD_RESERVED,
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_isp_scp_p1_cmd - ISP P1 IPI command strcture.
>>> + *
>>> + * @cmd_id: The IPI command ID.
>>> + * @init_param: The init formation for ISP_CMD_INIT.
>>> + * @config_param: The cmd configuration for ISP_CMD_CONFIG.
>>> + * @enabled_dmas: The meta configuration information for ISP_CMD_CONFIG_META.
>>> + * @is_stream_on: The stream information for ISP_CMD_STREAM.
>>> + * @ack_info: The cmd ack. information for ISP_CMD_ACK.
>>> + *
>>> + */
>>> +struct mtk_isp_scp_p1_cmd {
>>> +	u8 cmd_id;
>>> +	union {
>>> +		struct isp_init_info init_param;
>>> +		struct p1_config_param config_param;
>>> +		u32 enabled_dmas;
>>> +		struct P1_meta_frame meta_frame;
>>> +		u8 is_stream_on;
>>> +		struct isp_ack_info ack_info;
>>> +	};
>>> +} __packed;
>>> +
>>> +#endif /* __MTK_CAM_IPI_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>> new file mode 100644
>>> index 000000000000..ab2277f45fa4
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>> @@ -0,0 +1,95 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_REGS_H__
>>> +#define __MTK_CAM_REGS_H__
>>> +
>>> +/* ISP interrupt enable */
>>> +#define REG_CTL_RAW_INT_EN		0x0020
>>> +#define DMA_ERR_INT_EN			BIT(29)
>>> +
>>> +/* ISP interrupt status */
>>> +#define REG_CTL_RAW_INT_STAT		0x0024
>>> +#define VS_INT_ST			BIT(0)
>>> +#define TG_ERR_ST			BIT(4)
>>> +#define TG_GBERR_ST			BIT(5)
>>> +#define CQ_CODE_ERR_ST			BIT(6)
>>> +#define CQ_APB_ERR_ST			BIT(7)
>>> +#define CQ_VS_ERR_ST			BIT(8)
>>> +#define HW_PASS1_DON_ST			BIT(11)
>>> +#define SOF_INT_ST			BIT(12)
>>> +#define AMX_ERR_ST			BIT(15)
>>> +#define RMX_ERR_ST			BIT(16)
>>> +#define BMX_ERR_ST			BIT(17)
>>> +#define RRZO_ERR_ST			BIT(18)
>>> +#define AFO_ERR_ST			BIT(19)
>>> +#define IMGO_ERR_ST			BIT(20)
>>> +#define AAO_ERR_ST			BIT(21)
>>> +#define PSO_ERR_ST			BIT(22)
>>> +#define LCSO_ERR_ST			BIT(23)
>>> +#define BNR_ERR_ST			BIT(24)
>>> +#define LSCI_ERR_ST			BIT(25)
>>> +#define DMA_ERR_ST			BIT(29)
>>> +#define SW_PASS1_DON_ST			BIT(30)
>>> +
>>> +/* ISP interrupt 2 status */
>>> +#define REG_CTL_RAW_INT2_STAT		0x0034
>>> +#define AFO_DONE_ST			BIT(5)
>>> +#define AAO_DONE_ST			BIT(7)
>>> +
>>> +/* Configures sensor mode */
>>> +#define REG_TG_SEN_MODE			0x0230
>>> +#define TG_SEN_MODE_CMOS_EN		BIT(0)
>>> +
>>> +/* View finder mode control */
>>> +#define REG_TG_VF_CON			0x0234
>>> +#define TG_VF_CON_VFDATA_EN		BIT(0)
>>> +
>>> +/* View finder mode control */
>>> +#define REG_TG_INTER_ST			0x026c
>>> +#define TG_CS_MASK			0x3f00
>>> +#define TG_IDLE_ST			BIT(8)
>>> +
>>> +/* IMGO error status register */
>>> +#define REG_IMGO_ERR_STAT		0x1360
>>> +/* RRZO error status register */
>>> +#define REG_RRZO_ERR_STAT		0x1364
>>> +/* AAO error status register */
>>> +#define REG_AAO_ERR_STAT		0x1368
>>> +/* AFO error status register */
>>> +#define REG_AFO_ERR_STAT		0x136c
>>> +/* LCSO error status register */
>>> +#define REG_LCSO_ERR_STAT		0x1370
>>> +/* BPCI error status register */
>>> +#define REG_BPCI_ERR_STAT		0x137c
>>> +/* LSCI error status register */
>>> +#define REG_LSCI_ERR_STAT		0x1384
>>> +/* LMVO error status register */
>>> +#define REG_LMVO_ERR_STAT		0x1390
>>> +/* FLKO error status register */
>>> +#define REG_FLKO_ERR_STAT		0x1394
>>> +/* PSO error status register */
>>> +#define REG_PSO_ERR_STAT		0x13a0
>>> +
>>> +/* CQ0 base address */
>>> +#define REG_CQ_THR0_BASEADDR		0x0198
>>> +/* Frame sequence number */
>>> +#define REG_FRAME_SEQ_NUM		0x13b8
>>> +
>>> +/* IRQ Error Mask */
>>> +#define INT_ST_MASK_CAM_ERR		( \
>>> +					TG_ERR_ST |\
>>> +					TG_GBERR_ST |\
>>> +					CQ_CODE_ERR_ST |\
>>> +					CQ_APB_ERR_ST |\
>>> +					CQ_VS_ERR_ST |\
>>> +					BNR_ERR_ST |\
>>> +					RMX_ERR_ST |\
>>> +					BMX_ERR_ST |\
>>> +					BNR_ERR_ST |\
>>> +					LSCI_ERR_ST |\
>>> +					DMA_ERR_ST)
>>> +
>>
>> I would add a common prefix all the registers in the file.
>>
>> Also, add some docs to know what those acronyms means would be nice.
>>
> 
> Ok, add this in next patch.
> 
> 
>>> +#endif	/* __MTK_CAM_REGS_H__ */
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>> new file mode 100644
>>> index 000000000000..23fdb8b4abc5
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>> @@ -0,0 +1,2087 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +// Copyright (c) 2019 MediaTek Inc.
>>> +
>>> +#include <linux/device.h>
>>> +#include <linux/dma-mapping.h>
>>> +#include <linux/of.h>
>>> +#include <linux/of_graph.h>
>>> +#include <linux/of_platform.h>
>>> +#include <linux/module.h>
>>> +#include <linux/platform_device.h>
>>> +#include <linux/pm_runtime.h>
>>> +#include <linux/videodev2.h>
>>> +#include <media/media-entity.h>
>>> +#include <media/v4l2-async.h>
>>> +#include <media/v4l2-common.h>
>>> +#include <media/v4l2-event.h>
>>> +#include <media/v4l2-fwnode.h>
>>> +#include <media/v4l2-ioctl.h>
>>> +#include <media/v4l2-mc.h>
>>> +#include <media/v4l2-subdev.h>
>>> +#include <media/videobuf2-dma-contig.h>
>>
>> Please sort in alphabetical order.
>>
> 
> Fix in next patch
> 
>>> +
>>> +#include "mtk_cam.h"
>>> +#include "mtk_cam-hw.h"
>>> +
>>> +#define R_IMGO		BIT(0)
>>> +#define R_RRZO		BIT(1)
>>> +#define R_AAO		BIT(3)
>>> +#define R_AFO		BIT(4)
>>> +#define R_LCSO		BIT(5)
>>> +#define R_LMVO		BIT(7)
>>> +#define R_FLKO		BIT(8)
>>> +#define R_PSO		BIT(10)
>>
>> It would be nice to have better names of docs of what these means.
>>
> 
> Add in next patch
> 
>>> +
>>> +#define MTK_ISP_ONE_PIXEL_MODE		1
>>> +#define MTK_ISP_MIN_RESIZE_RATIO	6
>>> +#define MTK_ISP_MAX_RUNNING_JOBS	3
>>> +
>>> +#define MTK_CAM_CIO_PAD_SRC		4
>>> +#define MTK_CAM_CIO_PAD_SINK		11
>>> +
>>> +static inline struct mtk_cam_video_device *
>>> +file_to_mtk_cam_node(struct file *__file)
>>> +{
>>> +	return container_of(video_devdata(__file),
>>> +		struct mtk_cam_video_device, vdev);
>>> +}
>>> +
>>> +static inline struct mtk_cam_video_device *
>>> +mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
>>
>> no need for the underscore in __vq
>>
> 
> Revise in next patch
> 
>>> +{
>>> +	return container_of(__vq, struct mtk_cam_video_device, vbq);
>>> +}
>>> +
>>> +static inline struct mtk_cam_dev_request *
>>> +mtk_cam_req_to_dev_req(struct media_request *__req)
>>> +{
>>> +	return container_of(__req, struct mtk_cam_dev_request, req);
>>> +}
>>> +
>>> +static inline struct mtk_cam_dev_buffer *
>>> +mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
>>> +{
>>> +	return container_of(__vb, struct mtk_cam_dev_buffer, vbb.vb2_buf);
>>> +}
>>> +
>>> +static void mtk_cam_dev_job_done(struct mtk_cam_dev *cam,
>>> +				 struct mtk_cam_dev_request *req,
>>> +				 enum vb2_buffer_state state)
>>> +{
>>> +	struct media_request_object *obj, *obj_prev;
>>> +	unsigned long flags;
>>> +	u64 ts_eof = ktime_get_boottime_ns();
>>> +
>>> +	if (!cam->streaming)
>>
>> s/streaming/is_streaming
>>
>> this makes a bit more intuitive of what the the boolean means.
>>
> 
> Revise in next patch
> 
>>> +		return;
>>> +
>>> +	dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
>>> +		req->req.debug_str, req->frame_params.frame_seq_no, state);
>>> +
>>> +	list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
>>> +		struct vb2_buffer *vb;
>>> +		struct mtk_cam_dev_buffer *buf;
>>> +		struct mtk_cam_video_device *node;
>>> +
>>> +		if (!vb2_request_object_is_buffer(obj))
>>> +			continue;
>>> +		vb = container_of(obj, struct vb2_buffer, req_obj);
>>> +		buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +		node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +		spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +		list_del(&buf->list);
>>> +		spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +		buf->vbb.sequence = req->frame_params.frame_seq_no;
>>> +		if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
>>> +			vb->timestamp = ts_eof;
>>> +		else
>>> +			vb->timestamp = req->timestamp;
>>> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>> +	}
>>> +}
>>> +
>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>> +						unsigned int frame_seq_no)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>> +		dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
>>> +			req->frame_params.frame_seq_no, frame_seq_no);
>>> +
>>> +		/* Match by the en-queued request number */
>>> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
>>> +			spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +			return req;
>>> +		}
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>>> +				   unsigned int frame_seq_no)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>> +		dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>>> +			req->frame_params.frame_seq_no, frame_seq_no);
>>> +
>>> +		/* Match by the en-queued request number */
>>> +		if (req->frame_params.frame_seq_no == frame_seq_no) {
>>> +			cam->running_job_count--;
>>> +			/* Pass to user space */
>>> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>>> +			list_del(&req->list);
>>> +			break;
>>> +		} else if (req->frame_params.frame_seq_no < frame_seq_no) {
>>> +			cam->running_job_count--;
>>> +			/* Pass to user space for frame drop */
>>> +			mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>>> +			dev_warn(cam->dev, "frame_seq:%d drop\n",
>>> +				 req->frame_params.frame_seq_no);
>>
>> maybe a counter in debugfs instead of the warning.
>>
> 
> Do you mean to add counter to accumulate the total count of drop frames?

please see my comment above.

> Could we add this and also keep this warning message?

Userspace would still continue to work when this happens, not sure if it is worthy
adding a warn, I would move it to dev_dbg() instead IMHO.

> 
>>> +			list_del(&req->list);
>>> +		} else {
>>> +			break;
>>> +		}
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +}
>>> +
>>> +static void mtk_cam_dev_req_cleanup(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	dev_dbg(cam->dev, "%s\n", __func__);
>>> +
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list)
>>> +		list_del(&req->list);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list)
>>> +		list_del(&req->list);
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +}
>>> +
>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct mtk_cam_dev_request *req, *req_prev;
>>> +	unsigned long flags;
>>> +
>>> +	if (!cam->streaming) {
>>> +		dev_dbg(cam->dev, "stream is off\n");
>>> +		return;
>>> +	}
>>> +
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	spin_lock_irqsave(&cam->running_job_lock, flags);
>>> +	list_for_each_entry_safe(req, req_prev, &cam->pending_job_list, list) {
>>> +		if (cam->running_job_count >= MTK_ISP_MAX_RUNNING_JOBS) {
>>> +			dev_dbg(cam->dev, "jobs are full\n");
>>> +			break;
>>> +		}
>>> +		cam->running_job_count++;
>>> +		list_del(&req->list);
>>> +		list_add_tail(&req->list, &cam->running_job_list);
>>
>> list_move_tail() can be used.
>>
> 
> Revised in this patch.
> 
>>> +		mtk_isp_req_enqueue(cam, req);
>>> +	}
>>> +	spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +}
>>> +
>>> +static struct media_request *mtk_cam_req_alloc(struct media_device *mdev)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_dev_req;
>>> +
>>> +	cam_dev_req = kzalloc(sizeof(*cam_dev_req), GFP_KERNEL);
>>> +
>>> +	return &cam_dev_req->req;
>>> +}
>>> +
>>> +static void mtk_cam_req_free(struct media_request *req)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_dev_req = mtk_cam_req_to_dev_req(req);
>>> +
>>> +	kfree(cam_dev_req);
>>> +}
>>> +
>>> +static void mtk_cam_req_queue(struct media_request *req)
>>> +{
>>> +	struct mtk_cam_dev_request *cam_req = mtk_cam_req_to_dev_req(req);
>>> +	struct mtk_cam_dev *cam = container_of(req->mdev, struct mtk_cam_dev,
>>> +					       media_dev);
>>> +	unsigned long flags;
>>> +
>>> +	/* update frame_params's dma_bufs in mtk_cam_vb2_buf_queue */
>>> +	vb2_request_queue(req);
>>> +
>>> +	/* add to pending job list */
>>> +	spin_lock_irqsave(&cam->pending_job_lock, flags);
>>> +	list_add_tail(&cam_req->list, &cam->pending_job_list);
>>> +	spin_unlock_irqrestore(&cam->pending_job_lock, flags);
>>> +
>>> +	mtk_cam_dev_req_try_queue(cam);
>>> +}
>>> +
>>> +static unsigned int get_pixel_bits(unsigned int pix_fmt)
>>> +{
>>> +	switch (pix_fmt) {
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
>>> +		return 8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
>>> +		return 10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
>>> +		return 12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
>>> +		return 14;
>>> +	default:
>>> +		return 0;
>>> +	}
>>> +}
>>
>> which patchset are these pixel formats defined?
>> I couldn't find them in the ones you pointed.
>>
>> I also wonder if all of them need to be defined, or if the pre-defined ones can be used,
>> so you can use v4l2_format_info() to get the number of bytes.
>>
> 
> I miss some files related to pixel format definition in this patch set.
> You could refer the old patch set for pixel format definition.
> https://patchwork.kernel.org/patch/11126055/
> 
>>> +
>>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
>>> +			     struct v4l2_pix_format_mplane *mp)
>>> +{
>>> +	unsigned int bpl, ppl;
>>
>> bytes per line and pixels per line right?
>>
> 
> Yes.
> 
>>> +	unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>>
>> wouldn't be easier a get_pixel_bytes() function instead of bits?
>>
> 
> Sorry. I didn't get the point.
> The unit of return value is bits, not bytes.
> Do you suggest move bpl & ppl calculation into get_pixel_bits() and
> rename to get_pixel_bytes()?

Never mind, I misread it.

> 
>>> +	unsigned int width = mp->width;
>>> +
>>> +	bpl = 0;
>>> +	if (node_id == MTK_CAM_P1_MAIN_STREAM_OUT) {
>>> +		/* Bayer encoding format & 2 bytes alignment */
>>> +		bpl = ALIGN(DIV_ROUND_UP(width * pixel_bits, 8), 2);
>>> +	} else if (node_id == MTK_CAM_P1_PACKED_BIN_OUT) {
>>> +		/*
>>> +		 * The FULL-G encoding format
>>> +		 * 1 G component per pixel
>>> +		 * 1 R component per 4 pixel
>>> +		 * 1 B component per 4 pixel
>>> +		 * Total 4G/1R/1B in 4 pixel (pixel per line:ppl)
>>> +		 */
>>> +		ppl = DIV_ROUND_UP(width * 6, 4);
>>> +		bpl = DIV_ROUND_UP(ppl * pixel_bits, 8);
>>> +
>>> +		/* 4 bytes alignment for 10 bit & others are 8 bytes */
>>> +		if (pixel_bits == 10)
>>> +			bpl = ALIGN(bpl, 4);
>>> +		else
>>> +			bpl = ALIGN(bpl, 8);
>>> +	}
>>> +	/*
>>> +	 * This image output buffer will be input buffer of MTK CAM DIP HW
>>> +	 * For MTK CAM DIP HW constrained, it needs 4 bytes alignment
>>> +	 */
>>> +	bpl = ALIGN(bpl, 4);
>>> +
>>> +	mp->plane_fmt[0].bytesperline = bpl;
>>> +	mp->plane_fmt[0].sizeimage = bpl * mp->height;
>>> +
>>> +	dev_dbg(cam->dev, "node:%d width:%d bytesperline:%d sizeimage:%d\n",
>>> +		node_id, width, bpl, mp->plane_fmt[0].sizeimage);
>>> +}
>>> +
>>> +static const struct v4l2_format *
>>> +mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
>>> +{
>>> +	int i;
>>
>> unsigned
>>
> 
> Revised in next patch.
> 
>>> +	const struct v4l2_format *dev_fmt;
>>> +
>>> +	for (i = 0; i < desc->num_fmts; i++) {
>>> +		dev_fmt = &desc->fmts[i];
>>> +		if (dev_fmt->fmt.pix_mp.pixelformat == format)
>>> +			return dev_fmt;
>>> +	}
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +/* Get the default format setting */
>>> +static void
>>> +mtk_cam_dev_load_default_fmt(struct mtk_cam_dev *cam,
>>> +			     struct mtk_cam_dev_node_desc *queue_desc,
>>> +			     struct v4l2_format *dest)
>>> +{
>>> +	const struct v4l2_format *default_fmt =
>>> +		&queue_desc->fmts[queue_desc->default_fmt_idx];
>>> +
>>> +	dest->type = queue_desc->buf_type;
>>> +
>>> +	/* Configure default format based on node type */
>>> +	if (!queue_desc->image) {
>>> +		dest->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
>>> +		dest->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
>>> +		return;
>>> +	}
>>> +
>>> +	dest->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
>>> +	dest->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
>>> +	dest->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
>>> +	/* bytesperline & sizeimage calculation */
>>> +	cal_image_pix_mp(cam, queue_desc->id, &dest->fmt.pix_mp);
>>> +	dest->fmt.pix_mp.num_planes = 1;
>>> +
>>> +	dest->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>> +	dest->fmt.pix_mp.field = V4L2_FIELD_NONE;
>>> +	dest->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>> +	dest->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>> +	dest->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>> +}
>>> +
>>> +/* Utility functions */
>>> +static unsigned int get_sensor_pixel_id(unsigned int fmt)
>>> +{
>>> +	switch (fmt) {
>>> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
>>> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
>>> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
>>> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_B;
>>> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_GB;
>>> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_GR;
>>> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
>>> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
>>> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
>>> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
>>> +		return MTK_CAM_RAW_PXL_ID_R;
>>> +	default:
>>> +		return MTK_CAM_RAW_PXL_ID_UNKNOWN;
>>> +	}
>>> +}
>>> +
>>> +static unsigned int get_sensor_fmt(unsigned int fmt)
>>> +{
>>> +	switch (fmt) {
>>> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
>>> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
>>> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
>>> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
>>> +		return MTK_CAM_IMG_FMT_BAYER8;
>>> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
>>> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
>>> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
>>> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
>>> +		return MTK_CAM_IMG_FMT_BAYER10;
>>> +	case MEDIA_BUS_FMT_SBGGR12_1X12:
>>> +	case MEDIA_BUS_FMT_SGBRG12_1X12:
>>> +	case MEDIA_BUS_FMT_SGRBG12_1X12:
>>> +	case MEDIA_BUS_FMT_SRGGB12_1X12:
>>> +		return MTK_CAM_IMG_FMT_BAYER12;
>>> +	case MEDIA_BUS_FMT_SBGGR14_1X14:
>>> +	case MEDIA_BUS_FMT_SGBRG14_1X14:
>>> +	case MEDIA_BUS_FMT_SGRBG14_1X14:
>>> +	case MEDIA_BUS_FMT_SRGGB14_1X14:
>>> +		return MTK_CAM_IMG_FMT_BAYER14;
>>> +	default:
>>> +		return MTK_CAM_IMG_FMT_UNKNOWN;
>>> +	}
>>> +}
>>
>> I was wondering if it is not better to save all the media bus format
>> into a table, instead of having several swtch case statements.
>>
> 
> Ok, revise in next patch.
> 
>>> +
>>> +static unsigned int get_img_fmt(unsigned int fourcc)
>>> +{
>>> +	switch (fourcc) {
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8:
>>> +		return MTK_CAM_IMG_FMT_BAYER8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG8F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB8F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER8;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10:
>>> +		return MTK_CAM_IMG_FMT_BAYER10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG10F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB10F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER10;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12:
>>> +		return MTK_CAM_IMG_FMT_BAYER12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG12F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB12F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER12;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14:
>>> +		return MTK_CAM_IMG_FMT_BAYER14;
>>> +	case V4L2_PIX_FMT_MTISP_SBGGR14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGBRG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SGRBG14F:
>>> +	case V4L2_PIX_FMT_MTISP_SRGGB14F:
>>> +		return MTK_CAM_IMG_FMT_FG_BAYER14;
>>> +	default:
>>> +		return MTK_CAM_IMG_FMT_UNKNOWN;
>>> +	}> +}
>>
>> same for the pixelformat.
>>
>> Then you can cache object with the pixelformat in the main struct.
>>
> 
> Ok, revise in next patch.
> 
>>> +
>>> +static int config_img_fmt(struct mtk_cam_dev *cam, unsigned int node_id,
>>> +			  struct p1_img_output *out_fmt, int sd_width,
>>> +			  int sd_height)
>>> +{
>>> +	const struct v4l2_format *cfg_fmt = &cam->vdev_nodes[node_id].vdev_fmt;
>>> +
>>> +	/* Check output & input image size dimension */
>>> +	if (cfg_fmt->fmt.pix_mp.width > sd_width ||
>>> +	    cfg_fmt->fmt.pix_mp.height > sd_height) {
>>> +		dev_err(cam->dev, "node:%d cfg size is larger than sensor\n",
>>> +			node_id);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* Check resize ratio for resize out stream due to HW constraint */
>>> +	if (((cfg_fmt->fmt.pix_mp.width * 100 / sd_width) <
>>> +	    MTK_ISP_MIN_RESIZE_RATIO) ||
>>> +	    ((cfg_fmt->fmt.pix_mp.height * 100 / sd_height) <
>>> +	    MTK_ISP_MIN_RESIZE_RATIO)) {
>>> +		dev_err(cam->dev, "node:%d resize ratio is less than %d%%\n",
>>> +			node_id, MTK_ISP_MIN_RESIZE_RATIO);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	out_fmt->img_fmt = get_img_fmt(cfg_fmt->fmt.pix_mp.pixelformat);
>>> +	out_fmt->pixel_bits = get_pixel_bits(cfg_fmt->fmt.pix_mp.pixelformat);
>>> +	if (out_fmt->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
>>> +	    !out_fmt->pixel_bits) {
>>> +		dev_err(cam->dev, "node:%d unknown pixel fmt:%d\n",
>>> +			node_id, cfg_fmt->fmt.pix_mp.pixelformat);
>>> +		return -EINVAL;
>>> +	}
>>> +	dev_dbg(cam->dev, "node:%d pixel_bits:%d img_fmt:0x%x\n",
>>> +		node_id, out_fmt->pixel_bits, out_fmt->img_fmt);
>>> +
>>> +	out_fmt->size.w = cfg_fmt->fmt.pix_mp.width;
>>> +	out_fmt->size.h = cfg_fmt->fmt.pix_mp.height;
>>> +	out_fmt->size.stride = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
>>> +	out_fmt->size.xsize = cfg_fmt->fmt.pix_mp.plane_fmt[0].bytesperline;
>>> +
>>> +	out_fmt->crop.left = 0;
>>> +	out_fmt->crop.top = 0;
>>> +	out_fmt->crop.width = sd_width;
>>> +	out_fmt->crop.height = sd_height;
>>> +
>>> +	dev_dbg(cam->dev,
>>> +		"node:%d size=%0dx%0d, stride:%d, xsize:%d, crop=%0dx%0d\n",
>>> +		node_id, out_fmt->size.w, out_fmt->size.h,
>>> +		out_fmt->size.stride, out_fmt->size.xsize,
>>> +		out_fmt->crop.width, out_fmt->crop.height);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_dev_init_stream(struct mtk_cam_dev *cam)
>>> +{
>>> +	int i;
>>> +
>>> +	cam->enabled_count = 0;
>>> +	cam->enabled_dmas = 0;
>>> +	cam->stream_count = 0;
>>> +	cam->running_job_count = 0;
>>> +
>>> +	/* Get the enabled meta DMA ports */
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>> +		if (!cam->vdev_nodes[i].enabled)
>>> +			continue;
>>> +		cam->enabled_count++;
>>> +		cam->enabled_dmas |= cam->vdev_nodes[i].desc.dma_port;
>>> +	}
>>> +
>>> +	dev_dbg(cam->dev, "%s:%d:0x%x\n", __func__, cam->enabled_count,
>>> +		cam->enabled_dmas);
>>> +}
>>> +
>>> +static int mtk_cam_dev_isp_config(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct p1_config_param config_param;
>>> +	struct cfg_in_param *cfg_in_param;
>>> +	struct v4l2_subdev_format sd_fmt;
>>> +	int sd_width, sd_height, sd_code;
>>
>> are this sd_* variables required? Can't sd_fmt be directly accessed?
>>
> 
> Ok, revised in next patch set.
> 
>>> +	unsigned int enabled_dma_ports = cam->enabled_dmas;
>>> +	int ret;
>>> +
>>> +	/* Get sensor format configuration */
>>> +	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>>> +	ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
>>> +	if (ret) {
>>> +		dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +	sd_width = sd_fmt.format.width;
>>> +	sd_height = sd_fmt.format.height;
>>> +	sd_code = sd_fmt.format.code;
>>> +	dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
>>> +		sd_code);
>>
>> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
>> it should be configured from userspace.
>>
> 
> Could you explain why?
> Moreover, how does configuration from user space?

IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305

If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).

Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
can be configured by userspace and part can't (which iirc should't be done in the media API).

Do you need to use v4l2_device_register_subdev_nodes() ?

Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
https://patchwork.kernel.org/cover/11463183/

which would be more appropriated if you don't want userspace to configure the whole pipeline.

> 
>>> +
>>> +	memset(&config_param, 0, sizeof(config_param));
>>> +
>>> +	/* Update cfg_in_param */
>>> +	cfg_in_param = &config_param.cfg_in_param;
>>> +	cfg_in_param->continuous = true;
>>> +	/* Fix to one pixel mode in default */
>>> +	cfg_in_param->pixel_mode = MTK_ISP_ONE_PIXEL_MODE;
>>> +	cfg_in_param->crop.width = sd_width;
>>> +	cfg_in_param->crop.height = sd_height;
>>> +	cfg_in_param->raw_pixel_id = get_sensor_pixel_id(sd_code);
>>> +	cfg_in_param->img_fmt = get_sensor_fmt(sd_code);
>>> +	if (cfg_in_param->img_fmt == MTK_CAM_IMG_FMT_UNKNOWN ||
>>> +	    cfg_in_param->raw_pixel_id == MTK_CAM_RAW_PXL_ID_UNKNOWN) {
>>> +		dev_err(dev, "unknown sd code:%d\n", sd_code);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* Update cfg_main_param */
>>> +	config_param.cfg_main_param.pure_raw = true;
>>> +	config_param.cfg_main_param.pure_raw_pack = true;
>>> +	ret = config_img_fmt(cam, MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +			     &config_param.cfg_main_param.output,
>>> +			     sd_width, sd_height);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	/* Update cfg_resize_param */
>>> +	if (enabled_dma_ports & R_RRZO) {
>>> +		ret = config_img_fmt(cam, MTK_CAM_P1_PACKED_BIN_OUT,
>>> +				     &config_param.cfg_resize_param.output,
>>> +				     sd_width, sd_height);
>>> +		if (ret)
>>> +			return ret;
>>> +	} else {
>>> +		config_param.cfg_resize_param.bypass = true;
>>> +	}
>>> +
>>> +	/* Update enabled_dmas */
>>> +	config_param.enabled_dmas = enabled_dma_ports;
>>> +	mtk_isp_hw_config(cam, &config_param);
>>> +	dev_dbg(dev, "%s done\n", __func__);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam,
>>> +				  unsigned int frame_seq_no)
>>> +{
>>> +	struct v4l2_event event = {
>>> +		.type = V4L2_EVENT_FRAME_SYNC,
>>> +		.u.frame_sync.frame_sequence = frame_seq_no,
>>> +	};
>>> +
>>> +	v4l2_event_queue(cam->subdev.devnode, &event);
>>> +}
>>> +
>>> +static struct v4l2_subdev *
>>> +mtk_cam_cio_get_active_sensor(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct media_device *mdev = cam->seninf->entity.graph_obj.mdev;
>>> +	struct device *dev = cam->dev;
>>> +	struct media_entity *entity;
>>> +	struct v4l2_subdev *sensor;
>>> +
>>> +	sensor = NULL;
>>> +	media_device_for_each_entity(entity, mdev) {
>>> +		dev_dbg(dev, "media entity: %s:0x%x:%d\n",
>>> +			entity->name, entity->function, entity->stream_count);
>>> +		if (entity->function == MEDIA_ENT_F_CAM_SENSOR &&
>>> +		    entity->stream_count) {
>>> +			sensor = media_entity_to_v4l2_subdev(entity);
>>> +			dev_dbg(dev, "sensor found: %s\n", entity->name);
>>> +			break;
>>> +		}
>>> +	}
>>> +
>>> +	if (!sensor)
>>> +		dev_err(dev, "no seninf connected\n");
>>> +
>>> +	return sensor;
>>> +}
>>> +
>>> +static int mtk_cam_cio_stream_on(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!cam->seninf) {
>>> +		dev_err(dev, "no seninf connected\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	/* Get active sensor from graph topology */
>>> +	cam->sensor = mtk_cam_cio_get_active_sensor(cam);
>>> +	if (!cam->sensor)
>>> +		return -ENODEV;
>>> +
>>> +	/* Seninf must stream on first */
>>> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream on %s:%d\n",
>>> +			cam->seninf->entity.name, ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream on %s:%d\n",
>>> +			cam->sensor->entity.name, ret);
>>> +		goto fail_seninf_off;
>>> +	}
>>> +
>>> +	ret = mtk_cam_dev_isp_config(cam);
>>> +	if (ret)
>>> +		goto fail_sensor_off;
>>> +
>>> +	cam->streaming = true;
>>> +	mtk_isp_stream(cam, 1);
>>> +	mtk_cam_dev_req_try_queue(cam);
>>> +	dev_dbg(dev, "streamed on Pass 1\n");
>>> +
>>> +	return 0;
>>> +
>>> +fail_sensor_off:
>>> +	v4l2_subdev_call(cam->sensor, video, s_stream, 0);
>>> +fail_seninf_off:
>>> +	v4l2_subdev_call(cam->seninf, video, s_stream, 0);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_cam_cio_stream_off(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	ret = v4l2_subdev_call(cam->sensor, video, s_stream, 0);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream off %s:%d\n",
>>> +			cam->sensor->entity.name, ret);
>>> +		return -EPERM;
>>
>> Why -EPERM ?
>>
> 
> Ok, we will return ret directly.
> 
>>> +	}
>>> +
>>> +	ret = v4l2_subdev_call(cam->seninf, video, s_stream, 0);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to stream off %s:%d\n",
>>> +			cam->seninf->entity.name, ret);
>>> +		return -EPERM;
>>> +	}
>>> +
>>> +	cam->streaming = false;
>>> +	mtk_isp_stream(cam, 0);
>>> +	mtk_isp_hw_release(cam);
>>> +
>>> +	dev_dbg(dev, "streamed off Pass 1\n");
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_sd_s_stream(struct v4l2_subdev *sd, int enable)
>>> +{
>>> +	struct mtk_cam_dev *cam = container_of(sd, struct mtk_cam_dev, subdev);
>>> +
>>> +	if (enable) {
>>> +		/* Align vb2_core_streamon design */
>>> +		if (cam->streaming) {
>>> +			dev_warn(cam->dev, "already streaming on\n");
>>
>> I think just dev_dbg is enough.
>>
> 
> Fix in next patch.
> 
>>> +			return 0;
>>> +		}
>>> +		return mtk_cam_cio_stream_on(cam);
>>> +	}
>>> +
>>> +	if (!cam->streaming) {
>>> +		dev_warn(cam->dev, "already streaming off\n");
>>
>> same here
>>
> 
> Fix in next patch.
> 
>>> +		return 0;
>>> +	}
>>> +	return mtk_cam_cio_stream_off(cam);
>>> +}
>>> +
>>> +static int mtk_cam_sd_subscribe_event(struct v4l2_subdev *subdev,
>>> +				      struct v4l2_fh *fh,
>>> +				      struct v4l2_event_subscription *sub)
>>> +{
>>> +	switch (sub->type) {
>>> +	case V4L2_EVENT_FRAME_SYNC:
>>> +		return v4l2_event_subscribe(fh, sub, 0, NULL);
>>> +	default:
>>> +		return -EINVAL;
>>> +	}
>>> +}
>>> +
>>> +static int mtk_cam_media_link_setup(struct media_entity *entity,
>>> +				    const struct media_pad *local,
>>> +				    const struct media_pad *remote, u32 flags)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(entity, struct mtk_cam_dev, subdev.entity);
>>> +	u32 pad = local->index;
>>> +
>>> +	dev_dbg(cam->dev, "%s: %d->%d flags:0x%x\n",
>>> +		__func__, pad, remote->index, flags);
>>> +
>>> +	/*
>>> +	 * The video nodes exposed by the driver have pads indexes
>>> +	 * from 0 to MTK_CAM_P1_TOTAL_NODES - 1.
>>> +	 */
>>> +	if (pad < MTK_CAM_P1_TOTAL_NODES)
>>> +		cam->dev_nodes[pad].enabled =
>>> +			!!(flags & MEDIA_LNK_FL_ENABLED);
>>
>> Can't you just check the state of the link in the pad instead of saving it in cam->vdev_nodes[pad].enabled ?
>>
> 
> Ok, revised in next patch.
> 
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct device *dev = cam->dev;
>>> +	unsigned long flags;
>>> +
>>> +	dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
>>> +		node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
>>> +
>>> +	/* added the buffer into the tracking list */
>>> +	spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +	list_add_tail(&buf->list, &node->buf_list);
>>> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +
>>> +	/* update buffer internal address */
>>> +	req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
>>> +	req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
>>
>> isn't it an issue if userspace queue two buffers for the same video device in the same request?
>>
>> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
>> will be at req->frame_params.dma_bufs[buf->node_id], no?
>>
>> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
>> buffer?
>>
>> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
>>
> 
> 1. yes, it is a issue if userspace queues two buffers for the same video
> device with the same request FD.
> 
> 2. All buffers which are belonged different to different video devices
> in the request list will be updated to req->frame_params.dma_bufs by
> buf->node_id.
> 
> 3. It is not allowed for userspace to queue partial buffers for all
> enabled video devices. If it happens, it may trigger DMA errors for this
> request.

So I guess you should implement a custom .req_validate() to enforce userspace follows this.

> 
>>> +}
>>> +
>>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct device *dev = cam->dev;
>>> +	struct mtk_cam_dev_buffer *buf;
>>> +	dma_addr_t addr;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	buf->node_id = node->id;
>>> +	buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
>>> +	buf->scp_addr = 0;
>>> +
>>> +	/* SCP address is only valid for meta input buffer */
>>> +	if (!node->desc.smem_alloc)
>>> +		return 0;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	/* Use coherent address to get iova address */
>>> +	addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
>>> +				DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +	if (dma_mapping_error(dev, addr)) {
>>> +		dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
>>> +		return -EFAULT;
>>> +	}
>>> +	buf->scp_addr = buf->daddr;
>>> +	buf->daddr = addr;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
>>> +	const struct v4l2_format *fmt = &node->vdev_fmt;
>>> +	unsigned int size;
>>> +
>>> +	if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>> +	    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
>>> +		size = fmt->fmt.meta.buffersize;
>>> +	else
>>> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>> +
>>> +	if (vb2_plane_size(vb, 0) < size) {
>>> +		dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
>>> +			vb2_plane_size(vb, 0), size);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
>>> +		if (vb2_get_plane_payload(vb, 0) != size) {
>>> +			dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
>>> +				vb2_get_plane_payload(vb, 0), size);
>>> +			return -EINVAL;
>>> +		}
>>> +		return 0;
>>> +	}
>>> +
>>> +	v4l2_buf->field = V4L2_FIELD_NONE;
>>> +	vb2_set_plane_payload(vb, 0, size);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +	struct mtk_cam_dev_buffer *buf;
>>> +	struct device *dev = cam->dev;
>>> +
>>> +	if (!node->desc.smem_alloc)
>>> +		return;
>>> +
>>> +	buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>> +	dma_unmap_page_attrs(dev, buf->daddr,
>>> +			     vb->planes[0].length,
>>> +			     DMA_BIDIRECTIONAL,
>>> +			     DMA_ATTR_SKIP_CPU_SYNC);
>>> +}
>>> +
>>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>> +
>>> +	dev_dbg(cam->dev, "%s\n", __func__);
>>> +}
>>> +
>>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
>>> +				   unsigned int *num_buffers,
>>> +				   unsigned int *num_planes,
>>> +				   unsigned int sizes[],
>>> +				   struct device *alloc_devs[])
>>> +{
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	unsigned int max_buffer_count = node->desc.max_buf_count;
>>> +	const struct v4l2_format *fmt = &node->vdev_fmt;
>>> +	unsigned int size;
>>> +
>>> +	/* Check the limitation of buffer size */
>>> +	if (max_buffer_count)
>>> +		*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
>>> +
>>> +	if (node->desc.smem_alloc)
>>> +		vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>>> +
>>> +	if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>> +	    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
>>> +		size = fmt->fmt.meta.buffersize;
>>> +	else
>>> +		size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>> +
>>> +	/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
>>> +	if (*num_planes) {
>>> +		if (sizes[0] < size || *num_planes != 1)
>>> +			return -EINVAL;
>>> +	} else {
>>> +		*num_planes = 1;
>>> +		sizes[0] = size;
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
>>> +					   struct mtk_cam_video_device *node,
>>> +					   enum vb2_buffer_state state)
>>> +{
>>> +	struct mtk_cam_dev_buffer *buf, *buf_prev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&node->buf_list_lock, flags);
>>> +	list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
>>> +		list_del(&buf->list);
>>> +		vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>> +	}
>>> +	spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>> +}
>>> +
>>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
>>> +				       unsigned int count)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!node->enabled) {
>>> +		dev_err(dev, "Node:%d is not enabled\n", node->id);
>>> +		ret = -ENOLINK;
>>> +		goto fail_ret_buf;
>>> +	}
>>> +
>>> +	mutex_lock(&cam->op_lock);
>>> +	/* Start streaming of the whole pipeline now*/
>>> +	if (!cam->pipeline.streaming_count) {
>>
>> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
>>
> 
> The check is designed to start the media pipeline when we start
> streaming on the first node. You could refer the detail in below link.
> 
> https://patchwork.kernel.org/patch/10985819/

right, ok, this is when enabling streaming from multiple nodes.

media_pipeline_start() is usually called for every stream that starts.

So cam->pipeline.streaming_count can reflect the number of streams enabled.

So maybe you don't need cam->stream_count.

> 
> 
>>> +		ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
>>> +		if (ret) {
>>> +			dev_err(dev, "failed to start pipeline:%d\n", ret);
>>> +			goto fail_unlock;
>>> +		}
>>> +		mtk_cam_dev_init_stream(cam);
>>> +		ret = mtk_isp_hw_init(cam);

Would it make sense to move this to s_stream in the ISP's subdevice ?

>>> +		if (ret) {
>>> +			dev_err(dev, "failed to init HW:%d\n", ret);
>>> +			goto fail_stop_pipeline;
>>> +		}
>>> +	}
>>> +
>>> +	/* Media links are fixed after media_pipeline_start */
>>> +	cam->stream_count++;
>>> +	dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
>>> +		cam->enabled_count);
>>> +	if (cam->stream_count < cam->enabled_count) {

I'm also wondering, since you need to wait for all the enabled video devices
to start streaming, shouldn't this be done inside a request? So you can enable
all of them at once?

Also, like this you wouldn't need to check enabled links to query for enabled video
nodes, you can just enable the ones in the request.

make sense?

>>> +		mutex_unlock(&cam->op_lock);
>>> +		return 0;
>>> +	}
>>> +
>>> +	/* Stream on sub-devices node */
>>> +	ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
>>> +	if (ret)
>>> +		goto fail_no_stream;
>>> +	mutex_unlock(&cam->op_lock);
>>> +
>>> +	return 0;
>>> +
>>> +fail_no_stream:
>>> +	cam->stream_count--;
>>> +fail_stop_pipeline:
>>> +	if (cam->stream_count == 0)
>>> +		media_pipeline_stop(&node->vdev.entity);
>>> +fail_unlock:
>>> +	mutex_unlock(&cam->op_lock);
>>> +fail_ret_buf:
>>> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
>>> +{
>>> +	struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>> +	struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>> +	struct device *dev = cam->dev;
>>> +
>>> +	mutex_lock(&cam->op_lock);
>>> +	dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
>>> +		cam->stream_count);
>>> +	/* Check the first node to stream-off */
>>> +	if (cam->stream_count == cam->enabled_count)
>>> +		v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
>>> +
>>> +	mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>>> +	cam->stream_count--;
>>> +	if (cam->stream_count) {
>>> +		mutex_unlock(&cam->op_lock);
>>> +		return;
>>> +	}
>>> +	mutex_unlock(&cam->op_lock);
>>> +
>>> +	mtk_cam_dev_req_cleanup(cam);
>>> +	media_pipeline_stop(&node->vdev.entity);
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
>>> +				   struct v4l2_capability *cap)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +
>>> +	strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
>>> +	strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
>>> +	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
>>> +		 dev_name(cam->dev));
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
>>> +				   struct v4l2_fmtdesc *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (f->index >= node->desc.num_fmts)
>>> +		return -EINVAL;
>>> +
>>> +	/* f->description is filled in v4l_fill_fmtdesc function */
>>> +	f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
>>> +	f->flags = 0;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
>>> +				struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	f->fmt = node->vdev_fmt.fmt;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
>>> +				  struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +	struct device *dev = cam->dev;
>>> +	const struct v4l2_format *dev_fmt;
>>> +	struct v4l2_format try_fmt;
>>> +
>>> +	memset(&try_fmt, 0, sizeof(try_fmt));
>>> +	try_fmt.type = f->type;
>>> +
>>> +	/* Validate pixelformat */
>>> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
>>> +	if (!dev_fmt) {
>>> +		dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
>>> +		dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
>>> +	}
>>> +	try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
>>> +
>>> +	/* Validate image width & height range */
>>> +	try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
>>> +					     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
>>> +	try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
>>> +					      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
>>> +	/* 4 bytes alignment for width */
>>> +	try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
>>> +
>>> +	/* Only support one plane */
>>> +	try_fmt.fmt.pix_mp.num_planes = 1;
>>> +
>>> +	/* bytesperline & sizeimage calculation */
>>> +	cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
>>> +
>>> +	/* Constant format fields */
>>> +	try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>> +	try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
>>> +	try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>> +	try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>> +	try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>> +
>>> +	*f = try_fmt;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
>>> +				struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_dev *cam = video_drvdata(file);
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (vb2_is_busy(node->vdev.queue)) {
>>> +		dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
>>> +		return -EBUSY;
>>> +	}
>>> +
>>> +	/* Get the valid format */
>>> +	mtk_cam_vidioc_try_fmt(file, fh, f);
>>> +	/* Configure to video device */
>>> +	node->vdev_fmt = *f;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
>>> +					  struct v4l2_frmsizeenum *sizes)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
>>> +	const struct v4l2_format *dev_fmt;
>>> +
>>> +	dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
>>> +	if (!dev_fmt || sizes->index)
>>> +		return -EINVAL;
>>> +
>>> +	sizes->type = node->desc.frmsizes->type;
>>> +	memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
>>> +	       sizeof(sizes->stepwise));
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
>>> +					struct v4l2_fmtdesc *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	if (f->index)
>>> +		return -EINVAL;
>>> +
>>> +	/* f->description is filled in v4l_fill_fmtdesc function */
>>> +	f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
>>> +	f->flags = 0;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
>>> +				     struct v4l2_format *f)
>>> +{
>>> +	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>> +
>>> +	f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
>>> +	f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
>>> +	.subscribe_event = mtk_cam_sd_subscribe_event,
>>> +	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
>>> +};
>>> +
>>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
>>> +	.s_stream =  mtk_cam_sd_s_stream,
>>> +};
>>> +
>>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
>>> +	.core = &mtk_cam_subdev_core_ops,
>>> +	.video = &mtk_cam_subdev_video_ops,
>>> +};
>>
>> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
>> I wonder if pad ops shouldn't be implemented too (to be verified).
>>
> 
> Ok, I will investigate this.
> 
>>> +
>>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
>>> +	.link_setup = mtk_cam_media_link_setup,
>>> +	.link_validate = v4l2_subdev_link_validate,
>>> +};
>>> +
>>> +static const struct vb2_ops mtk_cam_vb2_ops = {
>>> +	.queue_setup = mtk_cam_vb2_queue_setup,
>>> +	.wait_prepare = vb2_ops_wait_prepare,
>>> +	.wait_finish = vb2_ops_wait_finish,
>>> +	.buf_init = mtk_cam_vb2_buf_init,
>>> +	.buf_prepare = mtk_cam_vb2_buf_prepare,
>>> +	.start_streaming = mtk_cam_vb2_start_streaming,
>>> +	.stop_streaming = mtk_cam_vb2_stop_streaming,
>>> +	.buf_queue = mtk_cam_vb2_buf_queue,
>>> +	.buf_cleanup = mtk_cam_vb2_buf_cleanup,
>>> +	.buf_request_complete = mtk_cam_vb2_request_complete,
>>> +};> +
>>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
>>> +	.unlocked_ioctl = video_ioctl2,
>>> +	.open = v4l2_fh_open,
>>> +	.release = vb2_fop_release,
>>> +	.poll = vb2_fop_poll,
>>> +	.mmap = vb2_fop_mmap,
>>> +#ifdef CONFIG_COMPAT
>>> +	.compat_ioctl32 = v4l2_compat_ioctl32,
>>> +#endif
>>> +};
>>> +
>>> +static const struct media_device_ops mtk_cam_media_ops = {
>>> +	.req_alloc = mtk_cam_req_alloc,
>>> +	.req_free = mtk_cam_req_free,
>>> +	.req_validate = vb2_request_validate,
>>> +	.req_queue = mtk_cam_req_queue,
>>> +};
>>> +
>>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
>>> +				  struct media_device *media_dev)
>>> +{
>>> +	/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
>>> +	unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
>>> +	struct device *dev = cam->dev;
>>> +	int i, ret;
>>> +
>>> +	media_dev->dev = cam->dev;
>>> +	strscpy(media_dev->model, dev_driver_string(dev),
>>> +		sizeof(media_dev->model));
>>> +	snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
>>> +		 "platform:%s", dev_name(dev));
>>> +	media_dev->hw_revision = 0;
>>> +	media_device_init(media_dev);
>>> +	media_dev->ops = &mtk_cam_media_ops;
>>> +
>>> +	ret = media_device_register(media_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register media device:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	/* Initialize subdev pads */
>>> +	cam->subdev_pads = devm_kcalloc(dev, num_pads,
>>> +					sizeof(*cam->subdev_pads),
>>> +					GFP_KERNEL);
>>> +	if (!cam->subdev_pads) {
>>> +		dev_err(dev, "failed to allocate subdev_pads\n");
>>> +		ret = -ENOMEM;
>>> +		goto fail_media_unreg;
>>> +	}
>>> +
>>> +	ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
>>> +				     cam->subdev_pads);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize media pads:%d\n", ret);
>>> +		goto fail_media_unreg;
>>> +	}
>>> +
>>> +	/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
>>> +	for (i = 0; i < num_pads; i++)
>>> +		cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>> +
>>> +	/* Customize the last one pad as CIO sink pad. */
>>> +	cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>> +
>>> +	return 0;
>>> +
>>> +fail_media_unreg:
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int
>>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
>>> +			      struct mtk_cam_video_device *node)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	struct video_device *vdev = &node->vdev;
>>> +	struct vb2_queue *vbq = &node->vbq;
>>> +	unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
>>> +	unsigned int link_flags = node->desc.link_flags;
>>> +	int ret;
>>> +
>>> +	/* Initialize mtk_cam_video_device */
>>> +	if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
>>> +		node->enabled = true;
>>> +	else
>>> +		node->enabled = false;
>>> +	mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
>>> +
>>> +	cam->subdev_pads[node->id].flags = output ?
>>> +		MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>> +
>>> +	/* Initialize media entities */
>>> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize media pad:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +	node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>> +
>>> +	/* Initialize vbq */
>>> +	vbq->type = node->desc.buf_type;
>>> +	if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
>>> +		vbq->io_modes = VB2_MMAP;
>>> +	else
>>> +		vbq->io_modes = VB2_MMAP | VB2_DMABUF;
>>> +
>>> +	if (node->desc.smem_alloc) {
>>> +		vbq->bidirectional = 1;
>>> +		vbq->dev = cam->smem_dev;
>>> +	} else {
>>> +		vbq->dev = dev;
>>> +	}
>>> +	vbq->ops = &mtk_cam_vb2_ops;
>>> +	vbq->mem_ops = &vb2_dma_contig_memops;
>>> +	vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
>>> +	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
>>> +	if (output)
>>> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
>>> +	else
>>> +		vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>>> +	/* No minimum buffers limitation */
>>> +	vbq->min_buffers_needed = 0;
>>> +	vbq->drv_priv = cam;
>>> +	vbq->lock = &node->vdev_lock;
>>> +	vbq->supports_requests = true;
>>> +	vbq->requires_requests = true;
>>> +
>>> +	ret = vb2_queue_init(vbq);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
>>> +		goto fail_media_clean;
>>> +	}
>>> +
>>> +	/* Initialize vdev */
>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s",
>>> +		 dev_driver_string(dev), node->desc.name);
>>> +	/* set cap/type/ioctl_ops of the video device */
>>> +	vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
>>> +	vdev->ioctl_ops = node->desc.ioctl_ops;
>>> +	vdev->fops = &mtk_cam_v4l2_fops;
>>> +	vdev->release = video_device_release_empty;
>>> +	vdev->lock = &node->vdev_lock;
>>> +	vdev->v4l2_dev = &cam->v4l2_dev;
>>> +	vdev->queue = &node->vbq;
>>> +	vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
>>> +	vdev->entity.function = MEDIA_ENT_F_IO_V4L;
>>> +	vdev->entity.ops = NULL;
>>> +	video_set_drvdata(vdev, cam);
>>> +	dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
>>> +
>>> +	/* Initialize miscellaneous variables */
>>> +	mutex_init(&node->vdev_lock);
>>> +	INIT_LIST_HEAD(&node->buf_list);
>>> +	spin_lock_init(&node->buf_list_lock);
>>> +
>>> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register vde:%d\n", ret);
>>> +		goto fail_vb2_rel;
>>> +	}
>>> +
>>> +	/* Create link between video node and the subdev pad */
>>> +	if (output) {
>>> +		ret = media_create_pad_link(&vdev->entity, 0,
>>> +					    &cam->subdev.entity,
>>> +					    node->id, link_flags);
>>> +	} else {
>>> +		ret = media_create_pad_link(&cam->subdev.entity,
>>> +					    node->id, &vdev->entity, 0,
>>> +					    link_flags);
>>> +	}
>>
>> No need for the curly braces.
>>
> 
> Revised in next patch.
> 
>>> +	if (ret)
>>> +		goto fail_vdev_ureg;
>>> +
>>> +	return 0;
>>> +
>>> +fail_vdev_ureg:
>>> +	video_unregister_device(vdev);
>>> +fail_vb2_rel:
>>> +	mutex_destroy(&node->vdev_lock);
>>> +	vb2_queue_release(vbq);
>>> +fail_media_clean:
>>> +	media_entity_cleanup(&vdev->entity);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void
>>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
>>> +{
>>> +	video_unregister_device(&node->vdev);
>>> +	vb2_queue_release(&node->vbq);
>>> +	media_entity_cleanup(&node->vdev.entity);
>>> +	mutex_destroy(&node->vdev_lock);
>>> +}
>>> +
>>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int i, ret;
>>> +
>>> +	/* Set up media device & pads */
>>> +	ret = mtk_cam_media_register(cam, &cam->media_dev);
>>> +	if (ret)
>>> +		return ret;
>>> +	dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
>>> +
>>> +	/* Set up v4l2 device */
>>> +	cam->v4l2_dev.mdev = &cam->media_dev;
>>> +	ret = v4l2_device_register(dev, &cam->v4l2_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register V4L2 device:%d\n", ret);
>>> +		goto fail_media_unreg;
>>> +	}
>>> +	dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
>>> +
>>> +	/* Initialize subdev */
>>> +	v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
>>> +	cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>>> +	cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
>>> +	cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>>> +				V4L2_SUBDEV_FL_HAS_EVENTS;
>>> +	snprintf(cam->subdev.name, sizeof(cam->subdev.name),
>>> +		 "%s", dev_driver_string(dev));
>>> +	v4l2_set_subdevdata(&cam->subdev, cam);
>>> +
>>> +	ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize subdev:%d\n", ret);
>>> +		goto fail_clean_media_entiy;
>>> +	}
>>> +	dev_dbg(dev, "registered %s\n", cam->subdev.name);
>>> +
>>> +	/* Create video nodes and links */
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>> +		struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
>>> +
>>> +		node->id = node->desc.id;
>>> +		ret = mtk_cam_video_register_device(cam, node);
>>> +		if (ret)
>>> +			goto fail_vdev_unreg;
>>> +	}
>>> +	vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
>>> +
>>> +	return 0;
>>> +
>>> +fail_vdev_unreg:
>>> +	for (i--; i >= 0; i--)
>>> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>> +fail_clean_media_entiy:
>>> +	media_entity_cleanup(&cam->subdev.entity);
>>> +	v4l2_device_unregister(&cam->v4l2_dev);
>>> +fail_media_unreg:
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
>>> +{
>>> +	int i;
>>> +
>>> +	for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
>>> +		mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>> +
>>> +	vb2_dma_contig_clear_max_seg_size(cam->dev);
>>> +	v4l2_device_unregister_subdev(&cam->subdev);
>>> +	v4l2_device_unregister(&cam->v4l2_dev);
>>> +	media_entity_cleanup(&cam->subdev.entity);
>>> +	media_device_unregister(&cam->media_dev);
>>> +	media_device_cleanup(&cam->media_dev);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
>>> +				      struct v4l2_subdev *sd,
>>> +				      struct v4l2_async_subdev *asd)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +
>>> +	if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>>> +		dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	cam->seninf = sd;
>>> +	dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
>>> +					struct v4l2_subdev *sd,
>>> +					struct v4l2_async_subdev *asd)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +
>>> +	cam->seninf = NULL;
>>> +	dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
>>> +}
>>> +
>>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
>>> +{
>>> +	struct mtk_cam_dev *cam =
>>> +		container_of(notifier, struct mtk_cam_dev, notifier);
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	if (!cam->seninf) {
>>> +		dev_err(dev, "No seninf subdev\n");
>>> +		return -ENODEV;
>>> +	}
>>> +
>>> +	ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
>>> +				    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
>>> +				    MEDIA_LNK_FL_IMMUTABLE |
>>> +				    MEDIA_LNK_FL_ENABLED);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to create pad link %s %s err:%d\n",
>>> +			cam->seninf->entity.name, cam->subdev.entity.name,
>>> +			ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
>>> +	.bound = mtk_cam_dev_notifier_bound,
>>> +	.unbind = mtk_cam_dev_notifier_unbind,
>>> +	.complete = mtk_cam_dev_notifier_complete,
>>> +};
>>> +
>>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
>>> +{
>>> +	struct device *dev = cam->dev;
>>> +	int ret;
>>> +
>>> +	v4l2_async_notifier_init(&cam->notifier);
>>> +	ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
>>> +		&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
>>
>> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
>>
>> Regards,
>> Helen
>>
> 
> Ok, we will investigate how to do.
> 
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	cam->notifier.ops = &mtk_cam_v4l2_async_ops;
>>> +	dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
>>> +	ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
>>> +	if (ret) {
>>> +		dev_err(dev, "failed to register async notifier : %d\n", ret);
>>> +		v4l2_async_notifier_cleanup(&cam->notifier);
>>> +	}
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
>>> +{
>>> +	v4l2_async_notifier_unregister(&cam->notifier);
>>> +	v4l2_async_notifier_cleanup(&cam->notifier);
>>> +}
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
>>> +	.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
>>> +	.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
>>> +	.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
>>> +	.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>> +};
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
>>> +	.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +};
>>> +
>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
>>> +	.vidioc_querycap = mtk_cam_vidioc_querycap,
>>> +	.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
>>> +	.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>> +	.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> +	.vidioc_streamon = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff = vb2_ioctl_streamoff,
>>> +	.vidioc_expbuf = vb2_ioctl_expbuf,
>>> +};> +
>>> +static const struct v4l2_format meta_fmts[] = {
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_PARAMS,
>>> +			.buffersize = 512 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_3A,
>>> +			.buffersize = 1200 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_AF,
>>> +			.buffersize = 640 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_LCS,
>>> +			.buffersize = 288 * SZ_1K,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.meta = {
>>> +			.dataformat = V4L2_META_FMT_MTISP_LMV,
>>> +			.buffersize = 256,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct v4l2_format stream_out_fmts[] = {
>>> +	/* This is a default image format */
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct v4l2_format bin_out_fmts[] = {
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
>>> +		},
>>> +	},
>>> +	{
>>> +		.fmt.pix_mp = {
>>> +			.width = IMG_MAX_WIDTH,
>>> +			.height = IMG_MAX_HEIGHT,
>>> +			.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
>>> +		},
>>> +	},
>>> +};
>>> +
>>> +static const struct
>>> +mtk_cam_dev_node_desc output_queues[] = {
>>> +	{
>>> +		.id = MTK_CAM_P1_META_IN_0,
>>> +		.name = "meta input",
>>> +		.cap = V4L2_CAP_META_OUTPUT,
>>> +		.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = true,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 0,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
>>> +	},
>>> +};
>>> +
>>> +static const struct
>>> +mtk_cam_dev_node_desc capture_queues[] = {
>>> +	{
>>> +		.id = MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +		.name = "main stream",
>>> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>> +		.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
>>> +		.image = true,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_IMGO,
>>> +		.fmts = stream_out_fmts,
>>> +		.num_fmts = ARRAY_SIZE(stream_out_fmts),
>>> +		.default_fmt_idx = 0,
>>> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>> +		.frmsizes = &(struct v4l2_frmsizeenum) {
>>> +			.index = 0,
>>> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>> +			.stepwise = {
>>> +				.max_width = IMG_MAX_WIDTH,
>>> +				.min_width = IMG_MIN_WIDTH,
>>> +				.max_height = IMG_MAX_HEIGHT,
>>> +				.min_height = IMG_MIN_HEIGHT,
>>> +				.step_height = 1,
>>> +				.step_width = 1,
>>> +			},
>>> +		},
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_PACKED_BIN_OUT,
>>> +		.name = "packed out",
>>> +		.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>> +		.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>> +		.link_flags = 0,
>>> +		.image = true,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_RRZO,
>>> +		.fmts = bin_out_fmts,
>>> +		.num_fmts = ARRAY_SIZE(bin_out_fmts),
>>> +		.default_fmt_idx = 0,
>>> +		.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>> +		.frmsizes = &(struct v4l2_frmsizeenum) {
>>> +			.index = 0,
>>> +			.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>> +			.stepwise = {
>>> +				.max_width = IMG_MAX_WIDTH,
>>> +				.min_width = IMG_MIN_WIDTH,
>>> +				.max_height = IMG_MAX_HEIGHT,
>>> +				.min_height = IMG_MIN_HEIGHT,
>>> +				.step_height = 1,
>>> +				.step_width = 1,
>>> +			},
>>> +		},
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_0,
>>> +		.name = "partial meta 0",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_AAO | R_FLKO | R_PSO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 1,
>>> +		.max_buf_count = 5,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_1,
>>> +		.name = "partial meta 1",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_AFO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 2,
>>> +		.max_buf_count = 5,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_2,
>>> +		.name = "partial meta 2",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_LCSO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 3,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +	{
>>> +		.id = MTK_CAM_P1_META_OUT_3,
>>> +		.name = "partial meta 3",
>>> +		.cap = V4L2_CAP_META_CAPTURE,
>>> +		.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>> +		.link_flags = 0,
>>> +		.image = false,
>>> +		.smem_alloc = false,
>>> +		.dma_port = R_LMVO,
>>> +		.fmts = meta_fmts,
>>> +		.default_fmt_idx = 4,
>>> +		.max_buf_count = 10,
>>> +		.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>> +	},
>>> +};
>>> +
>>> +/* The helper to configure the device context */
>>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
>>> +{
>>> +	unsigned int node_idx;
>>> +	int i;
>>> +
>>> +	node_idx = 0;
>>> +	/* Setup the output queue */
>>> +	for (i = 0; i < ARRAY_SIZE(output_queues); i++)
>>> +		cam->vdev_nodes[node_idx++].desc = output_queues[i];
>>> +
>>> +	/* Setup the capture queue */
>>> +	for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
>>> +		cam->vdev_nodes[node_idx++].desc = capture_queues[i];
>>> +}
>>> +
>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>> +		     struct mtk_cam_dev *cam)
>>> +{
>>> +	int ret;
>>> +
>>> +	cam->dev = &pdev->dev;
>>> +	mtk_cam_dev_queue_setup(cam);
>>> +
>>> +	spin_lock_init(&cam->pending_job_lock);
>>> +	spin_lock_init(&cam->running_job_lock);
>>> +	INIT_LIST_HEAD(&cam->pending_job_list);
>>> +	INIT_LIST_HEAD(&cam->running_job_list);
>>> +	mutex_init(&cam->op_lock);
>>> +
>>> +	/* v4l2 sub-device registration */
>>> +	ret = mtk_cam_v4l2_register(cam);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	ret = mtk_cam_v4l2_async_register(cam);
>>> +	if (ret)
>>> +		goto fail_v4l2_unreg;
>>> +
>>> +	return 0;
>>> +
>>> +fail_v4l2_unreg:
>>> +	mutex_destroy(&cam->op_lock);
>>> +	mtk_cam_v4l2_unregister(cam);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
>>> +{
>>> +	mtk_cam_v4l2_async_unregister(cam);
>>> +	mtk_cam_v4l2_unregister(cam);
>>> +	mutex_destroy(&cam->op_lock);
>>> +}
>>> +
>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>> new file mode 100644
>>> index 000000000000..0a340a1e65ea
>>> --- /dev/null
>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>> @@ -0,0 +1,244 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (c) 2019 MediaTek Inc.
>>> + */
>>> +
>>> +#ifndef __MTK_CAM_H__
>>> +#define __MTK_CAM_H__
>>> +
>>> +#include <linux/device.h>
>>> +#include <linux/types.h>
>>> +#include <linux/platform_device.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/videodev2.h>
>>> +#include <media/v4l2-device.h>
>>> +#include <media/v4l2-ctrls.h>
>>> +#include <media/v4l2-subdev.h>
>>> +#include <media/videobuf2-core.h>
>>> +#include <media/videobuf2-v4l2.h>
>>> +
>>> +#include "mtk_cam-ipi.h"
>>> +
>>> +#define IMG_MAX_WIDTH		5376
>>> +#define IMG_MAX_HEIGHT		4032
>>> +#define IMG_MIN_WIDTH		80
>>> +#define IMG_MIN_HEIGHT		60
>>> +
>>> +/*
>>> + * ID enum value for struct mtk_cam_dev_node_desc:id
>>> + * or mtk_cam_video_device:id
>>> + */
>>> +enum  {
>>> +	MTK_CAM_P1_META_IN_0 = 0,
>>> +	MTK_CAM_P1_MAIN_STREAM_OUT,
>>> +	MTK_CAM_P1_PACKED_BIN_OUT,
>>> +	MTK_CAM_P1_META_OUT_0,
>>> +	MTK_CAM_P1_META_OUT_1,
>>> +	MTK_CAM_P1_META_OUT_2,
>>> +	MTK_CAM_P1_META_OUT_3,
>>> +	MTK_CAM_P1_TOTAL_NODES
>>> +};
>>> +
>>> +/* Supported image format list */
>>> +#define MTK_CAM_IMG_FMT_UNKNOWN		0x0000
>>> +#define MTK_CAM_IMG_FMT_BAYER8		0x2200
>>> +#define MTK_CAM_IMG_FMT_BAYER10		0x2201
>>> +#define MTK_CAM_IMG_FMT_BAYER12		0x2202
>>> +#define MTK_CAM_IMG_FMT_BAYER14		0x2203
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER8	0x2204
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER10	0x2205
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER12	0x2206
>>> +#define MTK_CAM_IMG_FMT_FG_BAYER14	0x2207
>>> +
>>> +/* Supported bayer pixel order */
>>> +#define MTK_CAM_RAW_PXL_ID_B		0
>>> +#define MTK_CAM_RAW_PXL_ID_GB		1
>>> +#define MTK_CAM_RAW_PXL_ID_GR		2
>>> +#define MTK_CAM_RAW_PXL_ID_R		3
>>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN	4
>>> +
>>> +/*
>>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
>>> + *
>>> + * @frame_seq_no: The frame sequence of frame in driver layer.
>>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
>>> + *
>>> + */
>>> +struct mtk_p1_frame_param {
>>> +	unsigned int frame_seq_no;
>>> +	struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
>>> +} __packed;
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_request - MTK camera device request.
>>> + *
>>> + * @req: Embedded struct media request.
>>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
>>> + * @frame_work: work queue entry for frame transmission to SCP.
>>> + * @list: List entry of the object for @struct mtk_cam_dev:
>>> + *        pending_job_list or running_job_list.
>>> + * @timestamp: Start of frame timestamp in ns
>>> + *
>>> + */
>>> +struct mtk_cam_dev_request {
>>> +	struct media_request req;
>>> +	struct mtk_p1_frame_param frame_params;
>>> +	struct work_struct frame_work;
>>> +	struct list_head list;
>>> +	u64 timestamp;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
>>> + *
>>> + * @vbb: Embedded struct vb2_v4l2_buffer.
>>> + * @list: List entry of the object for @struct mtk_cam_video_device:
>>> + *        buf_list.
>>> + * @daddr: The DMA address of this buffer.
>>> + * @scp_addr: The SCP address of this buffer which
>>> + *            is only supported for meta input node.
>>> + * @node_id: The vidoe node id which this buffer belongs to.
>>> + *
>>> + */
>>> +struct mtk_cam_dev_buffer {
>>> +	struct vb2_v4l2_buffer vbb;
>>> +	struct list_head list;
>>> +	/* Intenal part */
>>> +	dma_addr_t daddr;
>>> +	dma_addr_t scp_addr;
>>> +	unsigned int node_id;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
>>> + *
>>> + * @id: id of the node
>>> + * @name: name of the node
>>> + * @cap: supported V4L2 capabilities
>>> + * @buf_type: supported V4L2 buffer type
>>> + * @dma_port: the dma ports associated to the node
>>> + * @link_flags: default media link flags
>>> + * @smem_alloc: using the smem_dev as alloc device or not
>>> + * @image: true for image node, false for meta node
>>> + * @num_fmts: the number of supported node formats
>>> + * @default_fmt_idx: default format of this node
>>> + * @max_buf_count: maximum VB2 buffer count
>>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
>>> + * @fmts: supported format
>>> + * @frmsizes: supported V4L2 frame size number
>>> + *
>>> + */
>>> +struct mtk_cam_dev_node_desc {
>>> +	u8 id;
>>> +	const char *name;
>>> +	u32 cap;
>>> +	u32 buf_type;
>>> +	u32 dma_port;
>>> +	u32 link_flags;
>>> +	u8 smem_alloc:1;
>>> +	u8 image:1;
>>> +	u8 num_fmts;
>>> +	u8 default_fmt_idx;
>>> +	u8 max_buf_count;
>>> +	const struct v4l2_ioctl_ops *ioctl_ops;
>>> +	const struct v4l2_format *fmts;
>>> +	const struct v4l2_frmsizeenum *frmsizes;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_video_device - Mediatek video device structure
>>> + *
>>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
>>> + * @enabled: Indicate the video device is enabled or not
>>> + * @desc: The node description of video device
>>> + * @vdev_fmt: The V4L2 format of video device
>>> + * @vdev_pad: The media pad graph object of video device
>>> + * @vbq: A videobuf queue of video device
>>> + * @vdev: The video device instance
>>> + * @vdev_lock: Serializes vb2 queue and video device operations
>>> + * @buf_list: List for enqueue buffers
>>> + * @buf_list_lock: Lock used to protect buffer list.
>>> + *
>>> + */
>>> +struct mtk_cam_video_device {
>>> +	unsigned int id;
>>> +	unsigned int enabled;
>>> +	struct mtk_cam_dev_node_desc desc;
>>> +	struct v4l2_format vdev_fmt;
>>> +	struct media_pad vdev_pad;
>>> +	struct vb2_queue vbq;
>>> +	struct video_device vdev;
>>> +	/* Serializes vb2 queue and video device operations */
>>> +	struct mutex vdev_lock;
>>> +	struct list_head buf_list;
>>> +	/* Lock used to protect buffer list */
>>> +	spinlock_t buf_list_lock;
>>> +};
>>> +
>>> +/*
>>> + * struct mtk_cam_dev - Mediatek camera device structure.
>>> + *
>>> + * @dev: Pointer to device.
>>> + * @smem_pdev: Pointer to shared memory device.
>>> + * @pipeline: Media pipeline information.
>>> + * @media_dev: Media device instance.
>>> + * @subdev: The V4L2 sub-device instance.
>>> + * @v4l2_dev: The V4L2 device driver instance.
>>> + * @notifier: The v4l2_device notifier data.
>>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
>>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
>>> + * @seninf: Pointer to the seninf sub-device.
>>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
>>> + * @streaming: Indicate the overall streaming status is on or off.
>>> + * @enabled_dmas: The enabled dma port information when streaming on.
>>> + * @enabled_count: Number of enabled video nodes
>>> + * @stream_count: Number of streaming video nodes
>>> + * @running_job_count: Nunber of running jobs in the HW driver.
>>> + * @pending_job_list: List to keep the media requests before en-queue into
>>> + *                    HW driver.
>>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
>>> + * @running_job_list: List to keep the media requests after en-queue into
>>> + *                    HW driver.
>>> + * @running_job_lock: Protect the running_job_list data.
>>> + * @op_lock: Serializes driver's VB2 callback operations.
>>> + *
>>> + */
>>> +struct mtk_cam_dev {
>>> +	struct device *dev;
>>> +	struct device *smem_dev;
>>> +	struct media_pipeline pipeline;
>>> +	struct media_device media_dev;
>>> +	struct v4l2_subdev subdev;
>>> +	struct v4l2_device v4l2_dev;
>>> +	struct v4l2_async_notifier notifier;
>>> +	struct media_pad *subdev_pads;
>>> +	struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
>>> +	struct v4l2_subdev *seninf;
>>> +	struct v4l2_subdev *sensor;
>>> +	unsigned int streaming;
>>> +	unsigned int enabled_dmas;
>>> +	unsigned int enabled_count;
>>> +	unsigned int stream_count;
>>> +	unsigned int running_job_count;
>>> +	struct list_head pending_job_list;
>>> +	/* Protect the pending_job_list data */
>>> +	spinlock_t pending_job_lock;
>>> +	struct list_head running_job_list;
>>> +	/* Protect the running_job_list data & running_job_count */
>>> +	spinlock_t running_job_lock;
>>> +	/* Serializes driver's VB2 callback operations */
>>> +	struct mutex op_lock;
>>> +};
>>> +
>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>> +		     struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
>>> +				   unsigned int frame_seq_no);
>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
>>> +				  unsigned int frame_seq_no);
>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>> +						unsigned int frame_seq_no);
>>> +
>>> +#endif /* __MTK_CAM_H__ */
>>>
>>
>> _______________________________________________
>> Linux-mediatek mailing list
>> Linux-mediatek@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 

Regards,
Helen

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
  2020-04-10 10:32       ` Jungo Lin
  (?)
@ 2020-04-14 12:25         ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-04-14 12:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: tfiga, hverkuil-cisco, laurent.pinchart, matthias.bgg, mchehab,
	shik, devicetree, Sean.Cheng, suleiman, Rynn.Wu, srv_heupstream,
	robh, ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media

Hi Jungo,

On 4/10/20 7:32 AM, Jungo Lin wrote:
> Hi Helen:
> 
> Thanks for your comment.
> 
> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>> Hi Jungo,
>>
>> I was taking a look at this patchset, please see my comments below.
>>
>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>> Hello,
>>>
>>> This patch series adding the driver for Pass 1 (P1) unit in
>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>> camera features of CrOS.
>>>
>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>> tuning parameters to adjust the image quality. It performs optical
>>> black correction, defect pixel correction, W/IR imbalance correction
>>> and lens shading correction for RAW processing.
>>>
>>> The driver is implemented with V4L2 and media controller framework so
>>> we have the following entities to describe the ISP pass 1 path.
>>>
>>> (The current metadata interface used in meta input and partial meta
>>> nodes is only a temporary solution to kick off the driver development
>>> and is not ready to be reviewed yet.)
>>>
>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>> It accepts the tuning buffer from user.
>>>
>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>> main stream and packed out video devices. When processing an image,
>>> Pass 1 hardware supports multiple output images with different sizes
>>> and formats so it needs two capture video devices ("main stream" and
>>> "packed out") to return the image data to the user.
>>>
>>> 3. main stream (capture video device): return the processed image data
>>> which is used in capture scenario.
>>>
>>> 4. packed out (capture video device): return the processed image data
>>> which is used in preview scenario.
>>>
>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>
>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>
>>> 7. partial meta 2 (capture video device): return the local contrast
>>>    enhanced statistics.
>>>
>>> 8. partial meta 3 (capture video device): return the local motion
>>>    vector statistics.
>>>
>>> The overall patches of the series is:
>>>
>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>   communication to compose ISP registers in the firmware.
>>>
>>> Here is ISP P1 media topology:
>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>> which are implemented in below patch[1][2][3][4]:
>>
>> I would be nice if you could provide a branch with those applied.
>>
> 
> We apply those patches in the chromeos-4.19 to test.
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> 
> 
>>>
>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>> patch sets.
>>>
>>> /usr/bin/media-ctl -p -d /dev/media2
>>>
>>> Media controller API version 4.19.89
>>>
>>> Media device information
>>> ------------------------
>>> driver          mtk-cam-p1
>>> model           mtk-cam-p1
>>> serial          
>>> bus info        platform:1a000000.camisp
>>> hw revision     0x0
>>> driver version  4.19.89
>>>
>>> Device topology
>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>
>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>> Is this correct?
>>
>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>
> 
> Currently, we only support single-cam in this SoC with upstream driver.
> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> we'd like to keep the naming to avoid confusion.

I suppose this new Mediatek SoC would use this same driver?
I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
naming will be different then, right? (I guess it's ok).

> 
>>>             type V4L2 subdev subtype Unknown flags 0
>>>             device node name /dev/v4l-subdev0
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1 meta input":0 []
>>
>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>
> 
> Ok, we will revise our naming in next patch.
> 
>>> 	pad1: Source
>>> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>
>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>
> 
> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> 
>>> 	pad2: Source
>>> 		-> "mtk-cam-p1 packed out":0 []
>>
>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>
> 
> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> padding in the memory. We may revise the naming in next patch.
> 
>>> 	pad3: Source
>>> 		-> "mtk-cam-p1 partial meta 0":0 []
>>> 	pad4: Source
>>> 		-> "mtk-cam-p1 partial meta 1":0 []
>>> 	pad5: Source
>>> 		-> "mtk-cam-p1 partial meta 2":0 []
>>> 	pad6: Source
>>> 		-> "mtk-cam-p1 partial meta 3":0 []
>>
>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>
>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>
>> I also would prefer to remove blank spaces.
>>
>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>
> 
> No, these links are optional to setup for userspace.

Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?

I'm not sure if using the link state is the best option (please see my comment on 5/5).
Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
interesting nodes at once.


Regards,
Helen

> For naming part, we will align with driver source codes.
> 
>>> 	pad7: Source
>>> 	pad8: Source
>>> 	pad9: Source
>>> 	pad10: Source
>>
>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>
> 
> These pads are just reserved purpose.
> We will plan to remove them in next patch.
> 
> Thanks,
> 
> Jungo
> 
>> Regards,
>> Helen
>>
>>> 	pad11: Sink
>>> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>
>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video2
>>> 	pad0: Source
>>> 		-> "mtk-cam-p1":0 []
>>>
>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video3
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video4
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":2 []
>>>
>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video5
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":3 []
>>>
>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video6
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":4 []
>>>
>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video7
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":5 []
>>>
>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video8
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":6 []
>>>
>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>              type V4L2 subdev subtype Unknown flags 0
>>>              device node name /dev/v4l-subdev1
>>> 	pad0: Sink
>>> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>> 		<- "ov8856 2-0010":0 [ENABLED]
>>> 	pad1: Sink
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		<- "ov02a10 4-003d":0 []
>>> 	pad2: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad3: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad4: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>> 	pad5: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad6: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad7: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad8: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad9: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad10: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad11: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>
>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev2
>>> 	pad0: Source
>>> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>> 		-> "1a040000.seninf":0 [ENABLED]
>>>
>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>              type V4L2 subdev subtype Lens flags 0
>>>              device node name /dev/v4l-subdev3
>>>
>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev4
>>> 	pad0: Source
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
>>> 		-> "1a040000.seninf":1 []
>>>
>>> ===========
>>> = history =
>>> ===========
>>>
>>> version 6:
>>>  - Add port node description in the dt-binding document and device tree
>>>    by Tomasz Figa.
>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>    version.
>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>    timeout event.
>>>  - Fix KE due to no sen-inf sub-device.
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>  - Add rst documents for Mediatek meta formats.
>>>  - New meta buffer structure design & re-factoring.
>>>
>>> version 5:
>>>  - Fixed Rob's comment on dt-binding format
>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>    and new timestamp type in driver
>>>  - Fix buffer en-queue timing issue in v4
>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 4:
>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>    patch[4]
>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 3:
>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>    reserved memory region. (Rob Herring)
>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>  - Revise ISP Pass1 Kconfig
>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>    declaration (Hans Verkuil)
>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>    in-dependently (Hans Verkuil)
>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Fix ISP de-initialize timing KE issue
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>
>>> Todo:
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>
>>> version 2:
>>>  - Add 3A enhancement feature which includes:
>>>    Separates 3A pipeline out of frame basis to improve
>>>    AE/AWB (exposure and white balance) performance.
>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>    and co-processor.
>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>  - Revised document wording for dt-bindings documents & dts information.
>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>    or add comments.
>>>    Revised buffer size for LMVO & LCSO.
>>>    Fix pixel format utility function.
>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>  - mtk_cam-v4l2-util.c
>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>    Revised stream on function for 3A enhancement feature
>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Add meta buffer index register definitions
>>>    Add meta DMA configuration function.
>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Add new meta strictures for 3A enhancement feature
>>>    Add new IPI command utility function for 3A enhancement feature
>>>    Enhance isp_composer_dma_sg_init function flow
>>>    Shorten overall IPI command structure size
>>>    Remove scp_state state checking
>>>    Improve code readability
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>    memory region.
>>>
>>> TODOs:
>>>  - 3A enhancement feature bug fixing
>>>
>>> version 1:
>>>  - Revised driver sources based on Tomasz's comments including
>>>    part1/2/3/4 in RFC V0 patch.
>>>  - Remove DMA cache mechanism.
>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>    features.
>>>  - Fixed v4l2-compliance test failure items.
>>>  - Add private controls for Mediatek camera middle-ware.
>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>    co-processor communication.
>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>    commands RX handling.
>>>  - Fix internal bugs.
>>>
>>> TODOs:
>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>    for shared memory management.
>>>  - Revised file names.
>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>
>>> version 0:
>>> - Initial submission
>>>
>>> ==================
>>>  Dependent patch set
>>> ==================
>>>
>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>> The patches are listed as following:
>>>
>>> [1]. media: support Mediatek sensor interface driver
>>> https://patchwork.kernel.org/cover/11145845/
>>>
>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>> https://patchwork.kernel.org/cover/11220785/
>>>
>>> [3]. media: i2c: Add support for OV02A10 sensor
>>> https://patchwork.kernel.org/cover/11284779/
>>>
>>> [4]. media: i2c: add support for DW9768 VCM driver
>>> https://patchwork.kernel.org/cover/11132299/
>>>
>>> [5]. Add support for mt8183 SCP
>>> https://patchwork.kernel.org/cover/11239065/
>>>
>>> [6]. MT8183 IOMMU SUPPORT
>>> https://patchwork.kernel.org/cover/11112765/
>>>
>>> ==================
>>>  Compliance test
>>> ==================
>>>
>>> The v4l2-compliance is built with the below lastest patch.
>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>
>>> Note 1.
>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>
>>> Note 2.
>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>
>>> Note 3.
>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>
>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>
>>> v4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           :
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media1 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.67
>>> 	Capabilities     : 0x8c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video25 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composiv4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media2 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x8c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video2 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000016
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000014 (20)
>>> 	Name             : mtk-cam-p1 main stream
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000015   : 0: Sink
>>> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video3 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300001c
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000001a (26)
>>> 	Name             : mtk-cam-p1 packed out
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100001b   : 0: Sink
>>> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video4 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000022
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000020 (32)
>>> 	Name             : mtk-cam-p1 partial meta 0
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000021   : 0: Sink
>>> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video5 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000028
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000026 (38)
>>> 	Name             : mtk-cam-p1 partial meta 1
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000027   : 0: Sink
>>> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video6 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300002e
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000002c (44)
>>> 	Name             : mtk-cam-p1 partial meta 2
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100002d   : 0: Sink
>>> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video7 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000034
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000032 (50)
>>> 	Name             : mtk-cam-p1 partial meta 3
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000033   : 0: Sink
>>> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video8 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000050
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000001 (1)
>>> 	Name             : mtk-cam-p1
>>> 	Function         : Video Pixel Formatter
>>> 	Pad 0x01000002   : 0: Sink
>>> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>> 	Pad 0x01000003   : 1: Source
>>> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>> 	Pad 0x01000004   : 2: Source
>>> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>> 	Pad 0x01000005   : 3: Source
>>> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>> 	Pad 0x01000006   : 4: Source
>>> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>> 	Pad 0x01000007   : 5: Source
>>> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>> 	Pad 0x01000008   : 6: Source
>>> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>> 	Pad 0x01000009   : 7: Source
>>> 	Pad 0x0100000a   : 8: Source
>>> 	Pad 0x0100000b   : 9: Source
>>> 	Pad 0x0100000c   : 10: Source
>>> 	Pad 0x0100000d   : 11: Sink
>>> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev0 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000052
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000038 (56)
>>> 	Name             : 1a040000.seninf
>>> 	Function         : Video Interface Bridge
>>> 	Pad 0x01000039   : 0: Sink
>>> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>> 	Pad 0x0100003a   : 1: Sink
>>> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>> 	Pad 0x0100003b   : 2: Sink
>>> 	Pad 0x0100003c   : 3: Sink
>>> 	Pad 0x0100003d   : 4: Source
>>> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>> 	Pad 0x0100003e   : 5: Source
>>> 	Pad 0x0100003f   : 6: Source
>>> 	Pad 0x01000040   : 7: Source
>>> 	Pad 0x01000041   : 8: Source
>>> 	Pad 0x01000042   : 9: Source
>>> 	Pad 0x01000043   : 10: Source
>>> 	Pad 0x01000044   : 11: Source
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev1 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000054
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000045 (69)
>>> 	Name             : ov8856 2-0010
>>> 	Function         : Camera Sensor
>>> 	Pad 0x01000046   : 0: Source
>>> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev2 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 11 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000056
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000049 (73)
>>> 	Name             : dw9768 2-000c
>>> 	Function         : Lens Controller
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev3 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000058
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x0000004a (74)
>>> 	Name             : ov02a10 4-003d
>>> 	Function         : Camera Sensor
>>> 	Pad 0x0100004b   : 0: Source
>>> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev4 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>> 	test VIDIOC_G/S_CTRL: FAIL
>>> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 10 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>
>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>
>>>
>>> Jungo Lin (5):
>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>   media: videodev2.h: Add new boottime timestamp type
>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>
>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>  drivers/media/platform/Kconfig                |    1 +
>>>  drivers/media/platform/Makefile               |    1 +
>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>  include/uapi/linux/videodev2.h                |   41 +
>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>
>>
>> _______________________________________________
>> Linux-mediatek mailing list
>> Linux-mediatek@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-04-14 12:25         ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-04-14 12:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, shik,
	tfiga, zwisler, ddavenport

Hi Jungo,

On 4/10/20 7:32 AM, Jungo Lin wrote:
> Hi Helen:
> 
> Thanks for your comment.
> 
> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>> Hi Jungo,
>>
>> I was taking a look at this patchset, please see my comments below.
>>
>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>> Hello,
>>>
>>> This patch series adding the driver for Pass 1 (P1) unit in
>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>> camera features of CrOS.
>>>
>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>> tuning parameters to adjust the image quality. It performs optical
>>> black correction, defect pixel correction, W/IR imbalance correction
>>> and lens shading correction for RAW processing.
>>>
>>> The driver is implemented with V4L2 and media controller framework so
>>> we have the following entities to describe the ISP pass 1 path.
>>>
>>> (The current metadata interface used in meta input and partial meta
>>> nodes is only a temporary solution to kick off the driver development
>>> and is not ready to be reviewed yet.)
>>>
>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>> It accepts the tuning buffer from user.
>>>
>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>> main stream and packed out video devices. When processing an image,
>>> Pass 1 hardware supports multiple output images with different sizes
>>> and formats so it needs two capture video devices ("main stream" and
>>> "packed out") to return the image data to the user.
>>>
>>> 3. main stream (capture video device): return the processed image data
>>> which is used in capture scenario.
>>>
>>> 4. packed out (capture video device): return the processed image data
>>> which is used in preview scenario.
>>>
>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>
>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>
>>> 7. partial meta 2 (capture video device): return the local contrast
>>>    enhanced statistics.
>>>
>>> 8. partial meta 3 (capture video device): return the local motion
>>>    vector statistics.
>>>
>>> The overall patches of the series is:
>>>
>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>   communication to compose ISP registers in the firmware.
>>>
>>> Here is ISP P1 media topology:
>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>> which are implemented in below patch[1][2][3][4]:
>>
>> I would be nice if you could provide a branch with those applied.
>>
> 
> We apply those patches in the chromeos-4.19 to test.
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> 
> 
>>>
>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>> patch sets.
>>>
>>> /usr/bin/media-ctl -p -d /dev/media2
>>>
>>> Media controller API version 4.19.89
>>>
>>> Media device information
>>> ------------------------
>>> driver          mtk-cam-p1
>>> model           mtk-cam-p1
>>> serial          
>>> bus info        platform:1a000000.camisp
>>> hw revision     0x0
>>> driver version  4.19.89
>>>
>>> Device topology
>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>
>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>> Is this correct?
>>
>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>
> 
> Currently, we only support single-cam in this SoC with upstream driver.
> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> we'd like to keep the naming to avoid confusion.

I suppose this new Mediatek SoC would use this same driver?
I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
naming will be different then, right? (I guess it's ok).

> 
>>>             type V4L2 subdev subtype Unknown flags 0
>>>             device node name /dev/v4l-subdev0
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1 meta input":0 []
>>
>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>
> 
> Ok, we will revise our naming in next patch.
> 
>>> 	pad1: Source
>>> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>
>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>
> 
> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> 
>>> 	pad2: Source
>>> 		-> "mtk-cam-p1 packed out":0 []
>>
>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>
> 
> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> padding in the memory. We may revise the naming in next patch.
> 
>>> 	pad3: Source
>>> 		-> "mtk-cam-p1 partial meta 0":0 []
>>> 	pad4: Source
>>> 		-> "mtk-cam-p1 partial meta 1":0 []
>>> 	pad5: Source
>>> 		-> "mtk-cam-p1 partial meta 2":0 []
>>> 	pad6: Source
>>> 		-> "mtk-cam-p1 partial meta 3":0 []
>>
>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>
>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>
>> I also would prefer to remove blank spaces.
>>
>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>
> 
> No, these links are optional to setup for userspace.

Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?

I'm not sure if using the link state is the best option (please see my comment on 5/5).
Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
interesting nodes at once.


Regards,
Helen

> For naming part, we will align with driver source codes.
> 
>>> 	pad7: Source
>>> 	pad8: Source
>>> 	pad9: Source
>>> 	pad10: Source
>>
>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>
> 
> These pads are just reserved purpose.
> We will plan to remove them in next patch.
> 
> Thanks,
> 
> Jungo
> 
>> Regards,
>> Helen
>>
>>> 	pad11: Sink
>>> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>
>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video2
>>> 	pad0: Source
>>> 		-> "mtk-cam-p1":0 []
>>>
>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video3
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video4
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":2 []
>>>
>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video5
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":3 []
>>>
>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video6
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":4 []
>>>
>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video7
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":5 []
>>>
>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video8
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":6 []
>>>
>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>              type V4L2 subdev subtype Unknown flags 0
>>>              device node name /dev/v4l-subdev1
>>> 	pad0: Sink
>>> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>> 		<- "ov8856 2-0010":0 [ENABLED]
>>> 	pad1: Sink
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		<- "ov02a10 4-003d":0 []
>>> 	pad2: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad3: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad4: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>> 	pad5: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad6: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad7: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad8: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad9: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad10: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad11: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>
>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev2
>>> 	pad0: Source
>>> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>> 		-> "1a040000.seninf":0 [ENABLED]
>>>
>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>              type V4L2 subdev subtype Lens flags 0
>>>              device node name /dev/v4l-subdev3
>>>
>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev4
>>> 	pad0: Source
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
>>> 		-> "1a040000.seninf":1 []
>>>
>>> ===========
>>> = history =
>>> ===========
>>>
>>> version 6:
>>>  - Add port node description in the dt-binding document and device tree
>>>    by Tomasz Figa.
>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>    version.
>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>    timeout event.
>>>  - Fix KE due to no sen-inf sub-device.
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>  - Add rst documents for Mediatek meta formats.
>>>  - New meta buffer structure design & re-factoring.
>>>
>>> version 5:
>>>  - Fixed Rob's comment on dt-binding format
>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>    and new timestamp type in driver
>>>  - Fix buffer en-queue timing issue in v4
>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 4:
>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>    patch[4]
>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 3:
>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>    reserved memory region. (Rob Herring)
>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>  - Revise ISP Pass1 Kconfig
>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>    declaration (Hans Verkuil)
>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>    in-dependently (Hans Verkuil)
>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Fix ISP de-initialize timing KE issue
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>
>>> Todo:
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>
>>> version 2:
>>>  - Add 3A enhancement feature which includes:
>>>    Separates 3A pipeline out of frame basis to improve
>>>    AE/AWB (exposure and white balance) performance.
>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>    and co-processor.
>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>  - Revised document wording for dt-bindings documents & dts information.
>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>    or add comments.
>>>    Revised buffer size for LMVO & LCSO.
>>>    Fix pixel format utility function.
>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>  - mtk_cam-v4l2-util.c
>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>    Revised stream on function for 3A enhancement feature
>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Add meta buffer index register definitions
>>>    Add meta DMA configuration function.
>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Add new meta strictures for 3A enhancement feature
>>>    Add new IPI command utility function for 3A enhancement feature
>>>    Enhance isp_composer_dma_sg_init function flow
>>>    Shorten overall IPI command structure size
>>>    Remove scp_state state checking
>>>    Improve code readability
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>    memory region.
>>>
>>> TODOs:
>>>  - 3A enhancement feature bug fixing
>>>
>>> version 1:
>>>  - Revised driver sources based on Tomasz's comments including
>>>    part1/2/3/4 in RFC V0 patch.
>>>  - Remove DMA cache mechanism.
>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>    features.
>>>  - Fixed v4l2-compliance test failure items.
>>>  - Add private controls for Mediatek camera middle-ware.
>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>    co-processor communication.
>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>    commands RX handling.
>>>  - Fix internal bugs.
>>>
>>> TODOs:
>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>    for shared memory management.
>>>  - Revised file names.
>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>
>>> version 0:
>>> - Initial submission
>>>
>>> ==================
>>>  Dependent patch set
>>> ==================
>>>
>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>> The patches are listed as following:
>>>
>>> [1]. media: support Mediatek sensor interface driver
>>> https://patchwork.kernel.org/cover/11145845/
>>>
>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>> https://patchwork.kernel.org/cover/11220785/
>>>
>>> [3]. media: i2c: Add support for OV02A10 sensor
>>> https://patchwork.kernel.org/cover/11284779/
>>>
>>> [4]. media: i2c: add support for DW9768 VCM driver
>>> https://patchwork.kernel.org/cover/11132299/
>>>
>>> [5]. Add support for mt8183 SCP
>>> https://patchwork.kernel.org/cover/11239065/
>>>
>>> [6]. MT8183 IOMMU SUPPORT
>>> https://patchwork.kernel.org/cover/11112765/
>>>
>>> ==================
>>>  Compliance test
>>> ==================
>>>
>>> The v4l2-compliance is built with the below lastest patch.
>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>
>>> Note 1.
>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>
>>> Note 2.
>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>
>>> Note 3.
>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>
>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>
>>> v4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           :
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media1 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.67
>>> 	Capabilities     : 0x8c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video25 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composiv4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media2 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x8c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video2 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000016
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000014 (20)
>>> 	Name             : mtk-cam-p1 main stream
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000015   : 0: Sink
>>> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video3 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300001c
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000001a (26)
>>> 	Name             : mtk-cam-p1 packed out
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100001b   : 0: Sink
>>> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video4 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000022
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000020 (32)
>>> 	Name             : mtk-cam-p1 partial meta 0
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000021   : 0: Sink
>>> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video5 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000028
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000026 (38)
>>> 	Name             : mtk-cam-p1 partial meta 1
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000027   : 0: Sink
>>> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video6 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300002e
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000002c (44)
>>> 	Name             : mtk-cam-p1 partial meta 2
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100002d   : 0: Sink
>>> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video7 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000034
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000032 (50)
>>> 	Name             : mtk-cam-p1 partial meta 3
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000033   : 0: Sink
>>> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video8 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000050
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000001 (1)
>>> 	Name             : mtk-cam-p1
>>> 	Function         : Video Pixel Formatter
>>> 	Pad 0x01000002   : 0: Sink
>>> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>> 	Pad 0x01000003   : 1: Source
>>> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>> 	Pad 0x01000004   : 2: Source
>>> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>> 	Pad 0x01000005   : 3: Source
>>> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>> 	Pad 0x01000006   : 4: Source
>>> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>> 	Pad 0x01000007   : 5: Source
>>> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>> 	Pad 0x01000008   : 6: Source
>>> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>> 	Pad 0x01000009   : 7: Source
>>> 	Pad 0x0100000a   : 8: Source
>>> 	Pad 0x0100000b   : 9: Source
>>> 	Pad 0x0100000c   : 10: Source
>>> 	Pad 0x0100000d   : 11: Sink
>>> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev0 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000052
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000038 (56)
>>> 	Name             : 1a040000.seninf
>>> 	Function         : Video Interface Bridge
>>> 	Pad 0x01000039   : 0: Sink
>>> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>> 	Pad 0x0100003a   : 1: Sink
>>> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>> 	Pad 0x0100003b   : 2: Sink
>>> 	Pad 0x0100003c   : 3: Sink
>>> 	Pad 0x0100003d   : 4: Source
>>> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>> 	Pad 0x0100003e   : 5: Source
>>> 	Pad 0x0100003f   : 6: Source
>>> 	Pad 0x01000040   : 7: Source
>>> 	Pad 0x01000041   : 8: Source
>>> 	Pad 0x01000042   : 9: Source
>>> 	Pad 0x01000043   : 10: Source
>>> 	Pad 0x01000044   : 11: Source
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev1 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000054
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000045 (69)
>>> 	Name             : ov8856 2-0010
>>> 	Function         : Camera Sensor
>>> 	Pad 0x01000046   : 0: Source
>>> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev2 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 11 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000056
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000049 (73)
>>> 	Name             : dw9768 2-000c
>>> 	Function         : Lens Controller
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev3 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000058
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x0000004a (74)
>>> 	Name             : ov02a10 4-003d
>>> 	Function         : Camera Sensor
>>> 	Pad 0x0100004b   : 0: Source
>>> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev4 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>> 	test VIDIOC_G/S_CTRL: FAIL
>>> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 10 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>
>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>
>>>
>>> Jungo Lin (5):
>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>   media: videodev2.h: Add new boottime timestamp type
>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>
>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>  drivers/media/platform/Kconfig                |    1 +
>>>  drivers/media/platform/Makefile               |    1 +
>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>  include/uapi/linux/videodev2.h                |   41 +
>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>
>>
>> _______________________________________________
>> Linux-mediatek mailing list
>> Linux-mediatek@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-04-14 12:25         ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-04-14 12:25 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, Rynn.Wu, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, matthias.bgg,
	mchehab, linux-arm-kernel, Sean.Cheng, srv_heupstream, shik,
	tfiga, zwisler, ddavenport

Hi Jungo,

On 4/10/20 7:32 AM, Jungo Lin wrote:
> Hi Helen:
> 
> Thanks for your comment.
> 
> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>> Hi Jungo,
>>
>> I was taking a look at this patchset, please see my comments below.
>>
>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>> Hello,
>>>
>>> This patch series adding the driver for Pass 1 (P1) unit in
>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>> camera features of CrOS.
>>>
>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>> tuning parameters to adjust the image quality. It performs optical
>>> black correction, defect pixel correction, W/IR imbalance correction
>>> and lens shading correction for RAW processing.
>>>
>>> The driver is implemented with V4L2 and media controller framework so
>>> we have the following entities to describe the ISP pass 1 path.
>>>
>>> (The current metadata interface used in meta input and partial meta
>>> nodes is only a temporary solution to kick off the driver development
>>> and is not ready to be reviewed yet.)
>>>
>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>> It accepts the tuning buffer from user.
>>>
>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>> main stream and packed out video devices. When processing an image,
>>> Pass 1 hardware supports multiple output images with different sizes
>>> and formats so it needs two capture video devices ("main stream" and
>>> "packed out") to return the image data to the user.
>>>
>>> 3. main stream (capture video device): return the processed image data
>>> which is used in capture scenario.
>>>
>>> 4. packed out (capture video device): return the processed image data
>>> which is used in preview scenario.
>>>
>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>
>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>
>>> 7. partial meta 2 (capture video device): return the local contrast
>>>    enhanced statistics.
>>>
>>> 8. partial meta 3 (capture video device): return the local motion
>>>    vector statistics.
>>>
>>> The overall patches of the series is:
>>>
>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>   communication to compose ISP registers in the firmware.
>>>
>>> Here is ISP P1 media topology:
>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>> which are implemented in below patch[1][2][3][4]:
>>
>> I would be nice if you could provide a branch with those applied.
>>
> 
> We apply those patches in the chromeos-4.19 to test.
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> 
> 
>>>
>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>> patch sets.
>>>
>>> /usr/bin/media-ctl -p -d /dev/media2
>>>
>>> Media controller API version 4.19.89
>>>
>>> Media device information
>>> ------------------------
>>> driver          mtk-cam-p1
>>> model           mtk-cam-p1
>>> serial          
>>> bus info        platform:1a000000.camisp
>>> hw revision     0x0
>>> driver version  4.19.89
>>>
>>> Device topology
>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>
>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>> Is this correct?
>>
>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>
> 
> Currently, we only support single-cam in this SoC with upstream driver.
> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> we'd like to keep the naming to avoid confusion.

I suppose this new Mediatek SoC would use this same driver?
I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
naming will be different then, right? (I guess it's ok).

> 
>>>             type V4L2 subdev subtype Unknown flags 0
>>>             device node name /dev/v4l-subdev0
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1 meta input":0 []
>>
>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>
> 
> Ok, we will revise our naming in next patch.
> 
>>> 	pad1: Source
>>> 		-> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>
>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>
> 
> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> 
>>> 	pad2: Source
>>> 		-> "mtk-cam-p1 packed out":0 []
>>
>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>
> 
> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> padding in the memory. We may revise the naming in next patch.
> 
>>> 	pad3: Source
>>> 		-> "mtk-cam-p1 partial meta 0":0 []
>>> 	pad4: Source
>>> 		-> "mtk-cam-p1 partial meta 1":0 []
>>> 	pad5: Source
>>> 		-> "mtk-cam-p1 partial meta 2":0 []
>>> 	pad6: Source
>>> 		-> "mtk-cam-p1 partial meta 3":0 []
>>
>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>
>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>
>> I also would prefer to remove blank spaces.
>>
>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>
> 
> No, these links are optional to setup for userspace.

Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?

I'm not sure if using the link state is the best option (please see my comment on 5/5).
Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
interesting nodes at once.


Regards,
Helen

> For naming part, we will align with driver source codes.
> 
>>> 	pad7: Source
>>> 	pad8: Source
>>> 	pad9: Source
>>> 	pad10: Source
>>
>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>
> 
> These pads are just reserved purpose.
> We will plan to remove them in next patch.
> 
> Thanks,
> 
> Jungo
> 
>> Regards,
>> Helen
>>
>>> 	pad11: Sink
>>> 		<- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>
>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video2
>>> 	pad0: Source
>>> 		-> "mtk-cam-p1":0 []
>>>
>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video3
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video4
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":2 []
>>>
>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video5
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":3 []
>>>
>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video6
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":4 []
>>>
>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video7
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":5 []
>>>
>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>              type Node subtype V4L flags 0
>>>              device node name /dev/video8
>>> 	pad0: Sink
>>> 		<- "mtk-cam-p1":6 []
>>>
>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>              type V4L2 subdev subtype Unknown flags 0
>>>              device node name /dev/v4l-subdev1
>>> 	pad0: Sink
>>> 		[fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>> 		<- "ov8856 2-0010":0 [ENABLED]
>>> 	pad1: Sink
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		<- "ov02a10 4-003d":0 []
>>> 	pad2: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad3: Sink
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad4: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 		-> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>> 	pad5: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad6: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad7: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad8: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad9: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad10: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>> 	pad11: Source
>>> 		[fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>
>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev2
>>> 	pad0: Source
>>> 		[fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>> 		-> "1a040000.seninf":0 [ENABLED]
>>>
>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>              type V4L2 subdev subtype Lens flags 0
>>>              device node name /dev/v4l-subdev3
>>>
>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev4
>>> 	pad0: Source
>>> 		[fmt:SRGGB10_1X10/1600x1200 field:none]
>>> 		-> "1a040000.seninf":1 []
>>>
>>> ===========
>>> = history =
>>> ===========
>>>
>>> version 6:
>>>  - Add port node description in the dt-binding document and device tree
>>>    by Tomasz Figa.
>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>    version.
>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>    timeout event.
>>>  - Fix KE due to no sen-inf sub-device.
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>  - Add rst documents for Mediatek meta formats.
>>>  - New meta buffer structure design & re-factoring.
>>>
>>> version 5:
>>>  - Fixed Rob's comment on dt-binding format
>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>    and new timestamp type in driver
>>>  - Fix buffer en-queue timing issue in v4
>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 4:
>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>    patch[4]
>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>
>>> Todo:
>>>  - vb2_ops's buf_request_complete callback function implementation
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>
>>> version 3:
>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>    reserved memory region. (Rob Herring)
>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>  - Revise ISP Pass1 Kconfig
>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>    declaration (Hans Verkuil)
>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>    in-dependently (Hans Verkuil)
>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Fix Drew's comments which are addressed in v2 patch
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>    Fix ISP de-initialize timing KE issue
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>
>>> Todo:
>>>  - Add rst documents for Mediatek meta formats
>>>  - New meta buffer structure design & re-factoring
>>>
>>> version 2:
>>>  - Add 3A enhancement feature which includes:
>>>    Separates 3A pipeline out of frame basis to improve
>>>    AE/AWB (exposure and white balance) performance.
>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>    and co-processor.
>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>  - Revised document wording for dt-bindings documents & dts information.
>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>    or add comments.
>>>    Revised buffer size for LMVO & LCSO.
>>>    Fix pixel format utility function.
>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>  - mtk_cam-v4l2-util.c
>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>    Revised stream on function for 3A enhancement feature
>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>    Add meta buffer index register definitions
>>>    Add meta DMA configuration function.
>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>    Add new meta strictures for 3A enhancement feature
>>>    Add new IPI command utility function for 3A enhancement feature
>>>    Enhance isp_composer_dma_sg_init function flow
>>>    Shorten overall IPI command structure size
>>>    Remove scp_state state checking
>>>    Improve code readability
>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>    memory region.
>>>
>>> TODOs:
>>>  - 3A enhancement feature bug fixing
>>>
>>> version 1:
>>>  - Revised driver sources based on Tomasz's comments including
>>>    part1/2/3/4 in RFC V0 patch.
>>>  - Remove DMA cache mechanism.
>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>    features.
>>>  - Fixed v4l2-compliance test failure items.
>>>  - Add private controls for Mediatek camera middle-ware.
>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>    co-processor communication.
>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>    commands RX handling.
>>>  - Fix internal bugs.
>>>
>>> TODOs:
>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>    for shared memory management.
>>>  - Revised file names.
>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>
>>> version 0:
>>> - Initial submission
>>>
>>> ==================
>>>  Dependent patch set
>>> ==================
>>>
>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>> The patches are listed as following:
>>>
>>> [1]. media: support Mediatek sensor interface driver
>>> https://patchwork.kernel.org/cover/11145845/
>>>
>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>> https://patchwork.kernel.org/cover/11220785/
>>>
>>> [3]. media: i2c: Add support for OV02A10 sensor
>>> https://patchwork.kernel.org/cover/11284779/
>>>
>>> [4]. media: i2c: add support for DW9768 VCM driver
>>> https://patchwork.kernel.org/cover/11132299/
>>>
>>> [5]. Add support for mt8183 SCP
>>> https://patchwork.kernel.org/cover/11239065/
>>>
>>> [6]. MT8183 IOMMU SUPPORT
>>> https://patchwork.kernel.org/cover/11112765/
>>>
>>> ==================
>>>  Compliance test
>>> ==================
>>>
>>> The v4l2-compliance is built with the below lastest patch.
>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>
>>> Note 1.
>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>
>>> Note 2.
>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>
>>> Note 3.
>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>
>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>
>>> v4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           :
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media1 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.67
>>> 	Capabilities     : 0x8c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.67
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.67
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video25 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composiv4l2-compliance SHA: not available, 32 bits
>>>
>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>>
>>> Required ioctls:
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/media2 open: OK
>>> 	test MEDIA_IOC_DEVICE_INFO: OK
>>> 	test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> 	test MEDIA_IOC_G_TOPOLOGY: OK
>>> 	Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> 	test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x8c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x0c200000
>>> 		Metadata Output
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000010
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000000e (14)
>>> 	Name             : mtk-cam-p1 meta input
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100000f   : 0: Source
>>> 	  Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video2 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000016
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000014 (20)
>>> 	Name             : mtk-cam-p1 main stream
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000015   : 0: Sink
>>> 	  Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video3 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04201000
>>> 		Video Capture Multiplanar
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300001c
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000001a (26)
>>> 	Name             : mtk-cam-p1 packed out
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100001b   : 0: Sink
>>> 	  Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video4 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000022
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000020 (32)
>>> 	Name             : mtk-cam-p1 partial meta 0
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000021   : 0: Sink
>>> 	  Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video5 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000028
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000026 (38)
>>> 	Name             : mtk-cam-p1 partial meta 1
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000027   : 0: Sink
>>> 	  Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video6 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x0300002e
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x0000002c (44)
>>> 	Name             : mtk-cam-p1 partial meta 2
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x0100002d   : 0: Sink
>>> 	  Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video7 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>
>>> Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Card type        : mtk-cam-p1
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Driver version   : 4.19.89
>>> 	Capabilities     : 0x84a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> 		Device Capabilities
>>> 	Device Caps      : 0x04a00000
>>> 		Metadata Capture
>>> 		Streaming
>>> 		Extended Pix Format
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000034
>>> 	Type             : V4L Video
>>> Entity Info:
>>> 	ID               : 0x00000032 (50)
>>> 	Name             : mtk-cam-p1 partial meta 3
>>> 	Function         : V4L2 I/O
>>> 	Pad 0x01000033   : 0: Sink
>>> 	  Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>> 	test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/video8 open: OK
>>> 	test VIDIOC_QUERYCAP: OK
>>> 	test VIDIOC_G/S_PRIORITY: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK
>>> 	test VIDIOC_TRY_FMT: OK
>>> 	test VIDIOC_S_FMT: OK
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 		fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>> 		fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>> 		fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> 		fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>> 		fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK
>>>
>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000050
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000001 (1)
>>> 	Name             : mtk-cam-p1
>>> 	Function         : Video Pixel Formatter
>>> 	Pad 0x01000002   : 0: Sink
>>> 	  Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>> 	Pad 0x01000003   : 1: Source
>>> 	  Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>> 	Pad 0x01000004   : 2: Source
>>> 	  Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>> 	Pad 0x01000005   : 3: Source
>>> 	  Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>> 	Pad 0x01000006   : 4: Source
>>> 	  Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>> 	Pad 0x01000007   : 5: Source
>>> 	  Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>> 	Pad 0x01000008   : 6: Source
>>> 	  Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>> 	Pad 0x01000009   : 7: Source
>>> 	Pad 0x0100000a   : 8: Source
>>> 	Pad 0x0100000b   : 9: Source
>>> 	Pad 0x0100000c   : 10: Source
>>> 	Pad 0x0100000d   : 11: Sink
>>> 	  Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev0 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000052
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000038 (56)
>>> 	Name             : 1a040000.seninf
>>> 	Function         : Video Interface Bridge
>>> 	Pad 0x01000039   : 0: Sink
>>> 	  Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>> 	Pad 0x0100003a   : 1: Sink
>>> 	  Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>> 	Pad 0x0100003b   : 2: Sink
>>> 	Pad 0x0100003c   : 3: Sink
>>> 	Pad 0x0100003d   : 4: Source
>>> 	  Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>> 	Pad 0x0100003e   : 5: Source
>>> 	Pad 0x0100003f   : 6: Source
>>> 	Pad 0x01000040   : 7: Source
>>> 	Pad 0x01000041   : 8: Source
>>> 	Pad 0x01000042   : 9: Source
>>> 	Pad 0x01000043   : 10: Source
>>> 	Pad 0x01000044   : 11: Source
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev1 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 1):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 2):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Sink Pad 3):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 4):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 5):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 6):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 7):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 8):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 9):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 10):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 11):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000054
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000045 (69)
>>> 	Name             : ov8856 2-0010
>>> 	Function         : Camera Sensor
>>> 	Pad 0x01000046   : 0: Source
>>> 	  Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev2 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 		fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>> 		fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 		fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>> 		fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 11 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000056
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x00000049 (73)
>>> 	Name             : dw9768 2-000c
>>> 	Function         : Lens Controller
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev3 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 	test VIDIOC_G/S_CTRL: OK
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>
>>> Media Driver Info:
>>> 	Driver name      : mtk-cam-p1
>>> 	Model            : mtk-cam-p1
>>> 	Serial           : 
>>> 	Bus info         : platform:1a000000.camisp
>>> 	Media version    : 4.19.89
>>> 	Hardware revision: 0x00000000 (0)
>>> 	Driver version   : 4.19.89
>>> Interface Info:
>>> 	ID               : 0x03000058
>>> 	Type             : V4L Sub-Device
>>> Entity Info:
>>> 	ID               : 0x0000004a (74)
>>> 	Name             : ov02a10 4-003d
>>> 	Function         : Camera Sensor
>>> 	Pad 0x0100004b   : 0: Source
>>> 	  Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>
>>> Required ioctls:
>>> 	test MC information (see 'Media Driver Info' above): OK
>>>
>>> Allow for multiple opens:
>>> 	test second /dev/v4l-subdev4 open: OK
>>> 	test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Sub-Device ioctls (Source Pad 0):
>>> 	test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>> 	test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>> 	test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>
>>> Control ioctls:
>>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> 	test VIDIOC_QUERYCTRL: OK
>>> 		fail: v4l2-test-controls.cpp(362): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>> 	test VIDIOC_G/S_CTRL: FAIL
>>> 		fail: v4l2-test-controls.cpp(549): returned control value out of range
>>> 		fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>> 		fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> 	Standard Controls: 10 Private Controls: 0
>>>
>>> Format ioctls:
>>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>>> 	test VIDIOC_G_FMT: OK (Not Supported)
>>> 	test VIDIOC_TRY_FMT: OK (Not Supported)
>>> 	test VIDIOC_S_FMT: OK (Not Supported)
>>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> 	test Cropping: OK (Not Supported)
>>> 	test Composing: OK (Not Supported)
>>> 	test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls:
>>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>> 	test VIDIOC_EXPBUF: OK (Not Supported)
>>> 	test Requests: OK (Not Supported)
>>>
>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>
>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>
>>>
>>> Jungo Lin (5):
>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>   media: videodev2.h: Add new boottime timestamp type
>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>
>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>  drivers/media/platform/Kconfig                |    1 +
>>>  drivers/media/platform/Makefile               |    1 +
>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>  include/uapi/linux/videodev2.h                |   41 +
>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>
>>
>> _______________________________________________
>> Linux-mediatek mailing list
>> Linux-mediatek@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
       [not found]           ` <b2c30e560e9b4ec488957ca62bae09fe@mtkmbs01n2.mediatek.inc>
  2020-05-04 12:27               ` Jungo Lin
@ 2020-05-04 12:27               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-05-04 12:27 UTC (permalink / raw)
  To: Helen Koike
  Cc: laurent.pinchart, matthias.bgg, mchehab, shik, devicetree,
	Sean.Cheng, suleiman, Pi-Hsun Shih, srv_heupstream, robh,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media, tfiga, hverkuil-cisco


Hi Helen;

Sorry for late reply.
Please check my feedback & questions below.

On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> On 4/8/20 11:05 PM, Jungo Lin wrote:
> > Hi Helen:
> >
> > Thanks for your comments.
> >
> > On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >> Hello Jungo,
> >>
> >> I was taking a look at this patch (thanks for the work),
> >> I didn't look in deep details, but I have some comments, please see
> >> below. I hope it helps.
> >>
> >> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>> This patch adds the Mediatek ISP P1 HW control device driver.
> >>> It handles the ISP HW configuration, provides interrupt handling and
> >>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> >>> implement standard V4L2 video driver that utilizes V4L2 and media
> >>> framework APIs. It supports one media device, one sub-device and
> >>> several video devices during initialization. Moreover, it also connects
> >>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> >>> co-process via SCP communication to compose ISP registers in the
> >>> firmware.
> >>>
> >>> (The current metadata interface used in meta input and partial
> >>> meta nodes is only a temporary solution to kick off the driver
> >>> development and is not ready to be reviewed yet.)
> >>>
> >>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> >>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> >>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> >>> ---
> >>> Changes from v6:
> >>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >>>  - Correct auto suspend timer value for suspend/resume issue
> >>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >>>  - Fix KE due to no sen-inf sub-device
> >>> ---
> >>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>
> >> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
> >> capture, which for metadata, and what is being used to deal with requests.
> >>
> >> It would make it easier to review imho.
> >>
> >
> > For file structure design, it was reviewed in the previous patch
> > serials.
> > e.g.
> > https://patchwork.kernel.org/patch/10938137/
> > If you think it is better, I will modify it.
> 
> Right, I saw a suggestion to merge two files there.
> 
> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
> So it reflects a bit the topology.
> But it is also up to you to see if it improves organization or not, it is just a suggestion.
> 

Ok, got your point.
We will discuss how to do internally.

[snip]
> >>> +isp_composer_hw_init(p1_dev);
> >>> +
> >>> +p1_dev->enqueued_frame_seq_no = 0;
> >>> +p1_dev->dequeued_frame_seq_no = 0;
> >>> +p1_dev->composed_frame_seq_no = 0;
> >>> +p1_dev->sof_count = 0;
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >>> +
> >>> +isp_composer_hw_deinit(p1_dev);
> >>> +pm_runtime_mark_last_busy(dev);
> >>> +pm_runtime_put_autosuspend(dev);
> >>> +rproc_shutdown(p1_dev->rproc_handle);
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >>> + struct mtk_cam_dev_request *req)
> >>> +{
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >>> +
> >>> +/* Accumulated frame sequence number */
> >>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> >>> +
> >>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >>> +queue_work(p1_dev->composer_wq, &req->frame_work);
> >>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no,
> >>> +cam->running_job_count);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> >>> +       unsigned int dequeued_frame_seq_no)
> >>> +{
> >>> +dma_addr_t base_addr = p1_dev->composer_iova;
> >>> +struct device *dev = p1_dev->dev;
> >>> +struct mtk_cam_dev_request *req;
> >>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> >>> +unsigned int addr_offset;
> >>> +
> >>> +/* Send V4L2_EVENT_FRAME_SYNC event */
> >>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +
> >>> +p1_dev->sof_count += 1;
> >>> +/* Save frame information */
> >>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> >>> +
> >>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +if (req)
> >>> +req->timestamp = ktime_get_boottime_ns();
> >>> +
> >>> +/* Update CQ base address if needed */
> >>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no);
> >>> +return;
> >>> +}
> >>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> >>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> >>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> >>> +{
> >>> +u32 val;
> >>> +
> >>> +dev_err(p1_dev->dev,
> >>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> >>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> >>> +dev_err(p1_dev->dev,
> >>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> >>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> >>
> >> I think if would be better to transfor those into dev_dbg and add a counter
> >> in debugfs.
> >>
> >
> > These error messages are important for debugging.
> > I suggest to keep in dev_err.
> 
> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
> end user not so much, since end users won't know the meaning of those values.
> 
> For end users a "dma failure" message would be enough, then advanced users can enable
> debug messages to see more.

OK.
Got your point.

> >
> > Moreover, could you give more information about debug counter?
> > I don't get your point.
> > Do you suggest to accumulate the total count of DMA errors?
> 
> 
> Yes, you could have a debugfs entry with error counters like:
> 
> cat /debugfs/mtk_isp/dma_err
> 8
> 
> So it is easier if this error happens very frequent or not.
> In the rkisp1 case we added:
> 
> /debugfs/rkisp1/data_loss
> /debugfs/rkisp1/pic_size_error
> /debugfs/rkisp1/mipi_error
> /debugfs/rkisp1/stats_error
> /debugfs/rkisp1/mp_stop_timeout
> /debugfs/rkisp1/sp_stop_timeout
> /debugfs/rkisp1/mp_frame_drop
> /debugfs/rkisp1/sp_frame_drop
> 
> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
> 
> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
> 

Ok, we will follow your suggestion.


[snip]

> >>> +return;
> >>> +
> >>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
> >>> +
> >>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> >>> +struct vb2_buffer *vb;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct mtk_cam_video_device *node;
> >>> +
> >>> +if (!vb2_request_object_is_buffer(obj))
> >>> +continue;
> >>> +vb = container_of(obj, struct vb2_buffer, req_obj);
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_del(&buf->list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> >>> +vb->timestamp = ts_eof;
> >>> +else
> >>> +vb->timestamp = req->timestamp;
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +}
> >>> +
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +return req;
> >>> +}
> >>> +}
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +
> >>> +return NULL;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> >>> +   unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> >>> +list_del(&req->list);
> >>> +break;
> >>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space for frame drop */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> >>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
> >>> + req->frame_params.frame_seq_no);
> >>
> >> maybe a counter in debugfs instead of the warning.
> >>
> >
> > Do you mean to add counter to accumulate the total count of drop frames?
> 
> please see my comment above.
> 

Ok, add this in next patch.

> > Could we add this and also keep this warning message?
> 
> Userspace would still continue to work when this happens, not sure if it is worthy
> adding a warn, I would move it to dev_dbg() instead IMHO.
> 

Ok, revise in next patch.

[snip]
> >>> +
> >>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> >>> +     struct v4l2_pix_format_mplane *mp)
> >>> +{
> >>> +unsigned int bpl, ppl;
> >>
> >> bytes per line and pixels per line right?
> >>
> >
> > Yes.
> >
> >>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >>
> >> wouldn't be easier a get_pixel_bytes() function instead of bits?
> >>
> >
> > Sorry. I didn't get the point.
> > The unit of return value is bits, not bytes.
> > Do you suggest move bpl & ppl calculation into get_pixel_bits() and
> > rename to get_pixel_bytes()?
> 
> Never mind, I misread it.
> 

Ok, we will skip this.

[snip]
> >>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
> >>> +int ret;
> >>> +
> >>> +/* Get sensor format configuration */
> >>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> >>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> >>> +if (ret) {
> >>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +sd_width = sd_fmt.format.width;
> >>> +sd_height = sd_fmt.format.height;
> >>> +sd_code = sd_fmt.format.code;
> >>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> >>> +sd_code);
> >>
> >> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
> >> it should be configured from userspace.
> >>
> >
> > Could you explain why?
> > Moreover, how does configuration from user space?
> 
> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
> 
> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
> 
> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
> can be configured by userspace and part can't (which iirc should't be done in the media API).
> 
> Do you need to use v4l2_device_register_subdev_nodes() ?
> 
> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
> https://patchwork.kernel.org/cover/11463183/
> 
> which would be more appropriated if you don't want userspace to configure the whole pipeline.
> 

The purpose of P1 sub-device is to provide V4L2 event subscribe with
V4L2_EVENT_FRAME_SYNC event for user space. It is the same
implementation as blow link.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853

As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
precisely.

[snip]

> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +unsigned long flags;
> >>> +
> >>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> >>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> >>> +
> >>> +/* added the buffer into the tracking list */
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_add_tail(&buf->list, &node->buf_list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +
> >>> +/* update buffer internal address */
> >>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> >>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> >>
> >> isn't it an issue if userspace queue two buffers for the same video device in the same request?
> >>
> >> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
> >> will be at req->frame_params.dma_bufs[buf->node_id], no?
> >>
> >> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
> >> buffer?
> >>
> >> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
> >>
> >
> > 1. yes, it is a issue if userspace queues two buffers for the same video
> > device with the same request FD.
> >
> > 2. All buffers which are belonged different to different video devices
> > in the request list will be updated to req->frame_params.dma_bufs by
> > buf->node_id.
> >
> > 3. It is not allowed for userspace to queue partial buffers for all
> > enabled video devices. If it happens, it may trigger DMA errors for this
> > request.
> 
> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
> 

For case 1, it is handled in the vb2_queue_or_prepare_buf.
https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

For case 3, I need to correct my previous answer. This behavior should
be OK for outputted DMA. We have frame buffer controller for each
outputted DMAs. So it means the tuning buffer node is mandatory for each
request, other nodes are optional. We will implement this
in .req_validate to check.

> >
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +dma_addr_t addr;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +buf->node_id = node->id;
> >>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> >>> +buf->scp_addr = 0;
> >>> +
> >>> +/* SCP address is only valid for meta input buffer */
> >>> +if (!node->desc.smem_alloc)
> >>> +return 0;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +/* Use coherent address to get iova address */
> >>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> >>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
> >>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> >>> +return -EFAULT;
> >>> +}
> >>> +buf->scp_addr = buf->daddr;
> >>> +buf->daddr = addr;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +if (vb2_plane_size(vb, 0) < size) {
> >>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> >>> +vb2_plane_size(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> >>> +if (vb2_get_plane_payload(vb, 0) != size) {
> >>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> >>> +vb2_get_plane_payload(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +return 0;
> >>> +}
> >>> +
> >>> +v4l2_buf->field = V4L2_FIELD_NONE;
> >>> +vb2_set_plane_payload(vb, 0, size);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +if (!node->desc.smem_alloc)
> >>> +return;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +dma_unmap_page_attrs(dev, buf->daddr,
> >>> +     vb->planes[0].length,
> >>> +     DMA_BIDIRECTIONAL,
> >>> +     DMA_ATTR_SKIP_CPU_SYNC);
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +
> >>> +dev_dbg(cam->dev, "%s\n", __func__);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> >>> +   unsigned int *num_buffers,
> >>> +   unsigned int *num_planes,
> >>> +   unsigned int sizes[],
> >>> +   struct device *alloc_devs[])
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +unsigned int max_buffer_count = node->desc.max_buf_count;
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +/* Check the limitation of buffer size */
> >>> +if (max_buffer_count)
> >>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> >>> +
> >>> +if (node->desc.smem_alloc)
> >>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> >>> +
> >>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> >>> +if (*num_planes) {
> >>> +if (sizes[0] < size || *num_planes != 1)
> >>> +return -EINVAL;
> >>> +} else {
> >>> +*num_planes = 1;
> >>> +sizes[0] = size;
> >>> +}
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> >>> +   struct mtk_cam_video_device *node,
> >>> +   enum vb2_buffer_state state)
> >>> +{
> >>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> >>> +list_del(&buf->list);
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> >>> +       unsigned int count)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!node->enabled) {
> >>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
> >>> +ret = -ENOLINK;
> >>> +goto fail_ret_buf;
> >>> +}
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +/* Start streaming of the whole pipeline now*/
> >>> +if (!cam->pipeline.streaming_count) {
> >>
> >> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
> >>
> >
> > The check is designed to start the media pipeline when we start
> > streaming on the first node. You could refer the detail in below link.
> >
> > https://patchwork.kernel.org/patch/10985819/
> 
> right, ok, this is when enabling streaming from multiple nodes.
> 
> media_pipeline_start() is usually called for every stream that starts.
> 
> So cam->pipeline.streaming_count can reflect the number of streams enabled.
> 
> So maybe you don't need cam->stream_count.
> 

Ok, revise in next patch.

> >
> >
> >>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
> >>> +goto fail_unlock;
> >>> +}
> >>> +mtk_cam_dev_init_stream(cam);
> >>> +ret = mtk_isp_hw_init(cam);
> 
> Would it make sense to move this to s_stream in the ISP's subdevice ?
> 

No, we like to initialize our ISP firmware here when the first video
node is streaming on. It is too late to initialize when all video nodes
are streaming-on.

> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init HW:%d\n", ret);
> >>> +goto fail_stop_pipeline;
> >>> +}
> >>> +}
> >>> +
> >>> +/* Media links are fixed after media_pipeline_start */
> >>> +cam->stream_count++;
> >>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> >>> +cam->enabled_count);
> >>> +if (cam->stream_count < cam->enabled_count) {
> 
> I'm also wondering, since you need to wait for all the enabled video devices
> to start streaming, shouldn't this be done inside a request? So you can enable
> all of them at once?
> 
> Also, like this you wouldn't need to check enabled links to query for enabled video
> nodes, you can just enable the ones in the request.
> 
> make sense?
> 

Sorry, I didn't get your point about this comment.
Which request could we handle this logic?
Do you mean move this logic into mtk_cam_req_queue function?

> >>> +mutex_unlock(&cam->op_lock);
> >>> +return 0;
> >>> +}
> >>> +
> >>> +/* Stream on sub-devices node */
> >>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> >>> +if (ret)
> >>> +goto fail_no_stream;
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_no_stream:
> >>> +cam->stream_count--;
> >>> +fail_stop_pipeline:
> >>> +if (cam->stream_count == 0)
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +fail_unlock:
> >>> +mutex_unlock(&cam->op_lock);
> >>> +fail_ret_buf:
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> >>> +cam->stream_count);
> >>> +/* Check the first node to stream-off */
> >>> +if (cam->stream_count == cam->enabled_count)
> >>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> >>> +
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >>> +cam->stream_count--;
> >>> +if (cam->stream_count) {
> >>> +mutex_unlock(&cam->op_lock);
> >>> +return;
> >>> +}
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +mtk_cam_dev_req_cleanup(cam);
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> >>> +   struct v4l2_capability *cap)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +
> >>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> >>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> >>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> >>> + dev_name(cam->dev));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> >>> +   struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index >= node->desc.num_fmts)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt = node->vdev_fmt.fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> >>> +  struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +struct device *dev = cam->dev;
> >>> +const struct v4l2_format *dev_fmt;
> >>> +struct v4l2_format try_fmt;
> >>> +
> >>> +memset(&try_fmt, 0, sizeof(try_fmt));
> >>> +try_fmt.type = f->type;
> >>> +
> >>> +/* Validate pixelformat */
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> >>> +if (!dev_fmt) {
> >>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> >>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> >>> +}
> >>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> >>> +
> >>> +/* Validate image width & height range */
> >>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> >>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> >>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> >>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> >>> +/* 4 bytes alignment for width */
> >>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> >>> +
> >>> +/* Only support one plane */
> >>> +try_fmt.fmt.pix_mp.num_planes = 1;
> >>> +
> >>> +/* bytesperline & sizeimage calculation */
> >>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> >>> +
> >>> +/* Constant format fields */
> >>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> >>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> >>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> >>> +
> >>> +*f = try_fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (vb2_is_busy(node->vdev.queue)) {
> >>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> >>> +return -EBUSY;
> >>> +}
> >>> +
> >>> +/* Get the valid format */
> >>> +mtk_cam_vidioc_try_fmt(file, fh, f);
> >>> +/* Configure to video device */
> >>> +node->vdev_fmt = *f;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> >>> +  struct v4l2_frmsizeenum *sizes)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> >>> +const struct v4l2_format *dev_fmt;
> >>> +
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> >>> +if (!dev_fmt || sizes->index)
> >>> +return -EINVAL;
> >>> +
> >>> +sizes->type = node->desc.frmsizes->type;
> >>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> >>> +       sizeof(sizes->stepwise));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> >>> +struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> >>> +     struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> >>> +.subscribe_event = mtk_cam_sd_subscribe_event,
> >>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> >>> +.s_stream =  mtk_cam_sd_s_stream,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> >>> +.core = &mtk_cam_subdev_core_ops,
> >>> +.video = &mtk_cam_subdev_video_ops,
> >>> +};
> >>
> >> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
> >> I wonder if pad ops shouldn't be implemented too (to be verified).
> >>
> >
> > Ok, I will investigate this.
> >
> >>> +
> >>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> >>> +.link_setup = mtk_cam_media_link_setup,
> >>> +.link_validate = v4l2_subdev_link_validate,
> >>> +};
> >>> +
> >>> +static const struct vb2_ops mtk_cam_vb2_ops = {
> >>> +.queue_setup = mtk_cam_vb2_queue_setup,
> >>> +.wait_prepare = vb2_ops_wait_prepare,
> >>> +.wait_finish = vb2_ops_wait_finish,
> >>> +.buf_init = mtk_cam_vb2_buf_init,
> >>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
> >>> +.start_streaming = mtk_cam_vb2_start_streaming,
> >>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
> >>> +.buf_queue = mtk_cam_vb2_buf_queue,
> >>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> >>> +.buf_request_complete = mtk_cam_vb2_request_complete,
> >>> +};> +
> >>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> >>> +.unlocked_ioctl = video_ioctl2,
> >>> +.open = v4l2_fh_open,
> >>> +.release = vb2_fop_release,
> >>> +.poll = vb2_fop_poll,
> >>> +.mmap = vb2_fop_mmap,
> >>> +#ifdef CONFIG_COMPAT
> >>> +.compat_ioctl32 = v4l2_compat_ioctl32,
> >>> +#endif
> >>> +};
> >>> +
> >>> +static const struct media_device_ops mtk_cam_media_ops = {
> >>> +.req_alloc = mtk_cam_req_alloc,
> >>> +.req_free = mtk_cam_req_free,
> >>> +.req_validate = vb2_request_validate,
> >>> +.req_queue = mtk_cam_req_queue,
> >>> +};
> >>> +
> >>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> >>> +  struct media_device *media_dev)
> >>> +{
> >>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> >>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +media_dev->dev = cam->dev;
> >>> +strscpy(media_dev->model, dev_driver_string(dev),
> >>> +sizeof(media_dev->model));
> >>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> >>> + "platform:%s", dev_name(dev));
> >>> +media_dev->hw_revision = 0;
> >>> +media_device_init(media_dev);
> >>> +media_dev->ops = &mtk_cam_media_ops;
> >>> +
> >>> +ret = media_device_register(media_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register media device:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +/* Initialize subdev pads */
> >>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
> >>> +sizeof(*cam->subdev_pads),
> >>> +GFP_KERNEL);
> >>> +if (!cam->subdev_pads) {
> >>> +dev_err(dev, "failed to allocate subdev_pads\n");
> >>> +ret = -ENOMEM;
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> >>> +     cam->subdev_pads);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> >>> +for (i = 0; i < num_pads; i++)
> >>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Customize the last one pad as CIO sink pad. */
> >>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int
> >>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> >>> +      struct mtk_cam_video_device *node)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct video_device *vdev = &node->vdev;
> >>> +struct vb2_queue *vbq = &node->vbq;
> >>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> >>> +unsigned int link_flags = node->desc.link_flags;
> >>> +int ret;
> >>> +
> >>> +/* Initialize mtk_cam_video_device */
> >>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> >>> +node->enabled = true;
> >>> +else
> >>> +node->enabled = false;
> >>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> >>> +
> >>> +cam->subdev_pads[node->id].flags = output ?
> >>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Initialize media entities */
> >>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> >>> +
> >>> +/* Initialize vbq */
> >>> +vbq->type = node->desc.buf_type;
> >>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> >>> +vbq->io_modes = VB2_MMAP;
> >>> +else
> >>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> >>> +
> >>> +if (node->desc.smem_alloc) {
> >>> +vbq->bidirectional = 1;
> >>> +vbq->dev = cam->smem_dev;
> >>> +} else {
> >>> +vbq->dev = dev;
> >>> +}
> >>> +vbq->ops = &mtk_cam_vb2_ops;
> >>> +vbq->mem_ops = &vb2_dma_contig_memops;
> >>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> >>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> >>> +if (output)
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> >>> +else
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> >>> +/* No minimum buffers limitation */
> >>> +vbq->min_buffers_needed = 0;
> >>> +vbq->drv_priv = cam;
> >>> +vbq->lock = &node->vdev_lock;
> >>> +vbq->supports_requests = true;
> >>> +vbq->requires_requests = true;
> >>> +
> >>> +ret = vb2_queue_init(vbq);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> >>> +goto fail_media_clean;
> >>> +}
> >>> +
> >>> +/* Initialize vdev */
> >>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> >>> + dev_driver_string(dev), node->desc.name);
> >>> +/* set cap/type/ioctl_ops of the video device */
> >>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> >>> +vdev->ioctl_ops = node->desc.ioctl_ops;
> >>> +vdev->fops = &mtk_cam_v4l2_fops;
> >>> +vdev->release = video_device_release_empty;
> >>> +vdev->lock = &node->vdev_lock;
> >>> +vdev->v4l2_dev = &cam->v4l2_dev;
> >>> +vdev->queue = &node->vbq;
> >>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> >>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> >>> +vdev->entity.ops = NULL;
> >>> +video_set_drvdata(vdev, cam);
> >>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> >>> +
> >>> +/* Initialize miscellaneous variables */
> >>> +mutex_init(&node->vdev_lock);
> >>> +INIT_LIST_HEAD(&node->buf_list);
> >>> +spin_lock_init(&node->buf_list_lock);
> >>> +
> >>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register vde:%d\n", ret);
> >>> +goto fail_vb2_rel;
> >>> +}
> >>> +
> >>> +/* Create link between video node and the subdev pad */
> >>> +if (output) {
> >>> +ret = media_create_pad_link(&vdev->entity, 0,
> >>> +    &cam->subdev.entity,
> >>> +    node->id, link_flags);
> >>> +} else {
> >>> +ret = media_create_pad_link(&cam->subdev.entity,
> >>> +    node->id, &vdev->entity, 0,
> >>> +    link_flags);
> >>> +}
> >>
> >> No need for the curly braces.
> >>
> >
> > Revised in next patch.
> >
> >>> +if (ret)
> >>> +goto fail_vdev_ureg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_ureg:
> >>> +video_unregister_device(vdev);
> >>> +fail_vb2_rel:
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +vb2_queue_release(vbq);
> >>> +fail_media_clean:
> >>> +media_entity_cleanup(&vdev->entity);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void
> >>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> >>> +{
> >>> +video_unregister_device(&node->vdev);
> >>> +vb2_queue_release(&node->vbq);
> >>> +media_entity_cleanup(&node->vdev.entity);
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +/* Set up media device & pads */
> >>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
> >>> +if (ret)
> >>> +return ret;
> >>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> >>> +
> >>> +/* Set up v4l2 device */
> >>> +cam->v4l2_dev.mdev = &cam->media_dev;
> >>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> >>> +
> >>> +/* Initialize subdev */
> >>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> >>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> >>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> >>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> >>> +V4L2_SUBDEV_FL_HAS_EVENTS;
> >>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> >>> + "%s", dev_driver_string(dev));
> >>> +v4l2_set_subdevdata(&cam->subdev, cam);
> >>> +
> >>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
> >>> +goto fail_clean_media_entiy;
> >>> +}
> >>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
> >>> +
> >>> +/* Create video nodes and links */
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> >>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> >>> +
> >>> +node->id = node->desc.id;
> >>> +ret = mtk_cam_video_register_device(cam, node);
> >>> +if (ret)
> >>> +goto fail_vdev_unreg;
> >>> +}
> >>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_unreg:
> >>> +for (i--; i >= 0; i--)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +fail_clean_media_entiy:
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +int i;
> >>> +
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +
> >>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
> >>> +v4l2_device_unregister_subdev(&cam->subdev);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> >>> +      struct v4l2_subdev *sd,
> >>> +      struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +cam->seninf = sd;
> >>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> >>> +struct v4l2_subdev *sd,
> >>> +struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +cam->seninf = NULL;
> >>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!cam->seninf) {
> >>> +dev_err(dev, "No seninf subdev\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> >>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> >>> +    MEDIA_LNK_FL_IMMUTABLE |
> >>> +    MEDIA_LNK_FL_ENABLED);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
> >>> +cam->seninf->entity.name, cam->subdev.entity.name,
> >>> +ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> >>> +.bound = mtk_cam_dev_notifier_bound,
> >>> +.unbind = mtk_cam_dev_notifier_unbind,
> >>> +.complete = mtk_cam_dev_notifier_complete,
> >>> +};
> >>> +
> >>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +v4l2_async_notifier_init(&cam->notifier);
> >>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> >>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> >>
> >> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
> >>
> >> Regards,
> >> Helen
> >>
> >
> > Ok, we will investigate how to do.
> >
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> >>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> >>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +v4l2_async_notifier_unregister(&cam->notifier);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> >>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> >>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> >>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> >>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> >>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};> +
> >>> +static const struct v4l2_format meta_fmts[] = {
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> >>> +.buffersize = 512 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_3A,
> >>> +.buffersize = 1200 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_AF,
> >>> +.buffersize = 640 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
> >>> +.buffersize = 288 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
> >>> +.buffersize = 256,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format stream_out_fmts[] = {
> >>> +/* This is a default image format */
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format bin_out_fmts[] = {
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc output_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_META_IN_0,
> >>> +.name = "meta input",
> >>> +.cap = V4L2_CAP_META_OUTPUT,
> >>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = true,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 0,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc capture_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +.name = "main stream",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_IMGO,
> >>> +.fmts = stream_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +.name = "packed out",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = 0,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_RRZO,
> >>> +.fmts = bin_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_0,
> >>> +.name = "partial meta 0",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AAO | R_FLKO | R_PSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 1,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_1,
> >>> +.name = "partial meta 1",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AFO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 2,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_2,
> >>> +.name = "partial meta 2",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LCSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 3,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_3,
> >>> +.name = "partial meta 3",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LMVO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 4,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +/* The helper to configure the device context */
> >>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +unsigned int node_idx;
> >>> +int i;
> >>> +
> >>> +node_idx = 0;
> >>> +/* Setup the output queue */
> >>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
> >>> +
> >>> +/* Setup the capture queue */
> >>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> >>> +}
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam)
> >>> +{
> >>> +int ret;
> >>> +
> >>> +cam->dev = &pdev->dev;
> >>> +mtk_cam_dev_queue_setup(cam);
> >>> +
> >>> +spin_lock_init(&cam->pending_job_lock);
> >>> +spin_lock_init(&cam->running_job_lock);
> >>> +INIT_LIST_HEAD(&cam->pending_job_list);
> >>> +INIT_LIST_HEAD(&cam->running_job_list);
> >>> +mutex_init(&cam->op_lock);
> >>> +
> >>> +/* v4l2 sub-device registration */
> >>> +ret = mtk_cam_v4l2_register(cam);
> >>> +if (ret)
> >>> +return ret;
> >>> +
> >>> +ret = mtk_cam_v4l2_async_register(cam);
> >>> +if (ret)
> >>> +goto fail_v4l2_unreg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_v4l2_unreg:
> >>> +mutex_destroy(&cam->op_lock);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +mtk_cam_v4l2_async_unregister(cam);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +mutex_destroy(&cam->op_lock);
> >>> +}
> >>> +
> >>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> new file mode 100644
> >>> index 000000000000..0a340a1e65ea
> >>> --- /dev/null
> >>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> @@ -0,0 +1,244 @@
> >>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>> +/*
> >>> + * Copyright (c) 2019 MediaTek Inc.
> >>> + */
> >>> +
> >>> +#ifndef __MTK_CAM_H__
> >>> +#define __MTK_CAM_H__
> >>> +
> >>> +#include <linux/device.h>
> >>> +#include <linux/types.h>
> >>> +#include <linux/platform_device.h>
> >>> +#include <linux/spinlock.h>
> >>> +#include <linux/videodev2.h>
> >>> +#include <media/v4l2-device.h>
> >>> +#include <media/v4l2-ctrls.h>
> >>> +#include <media/v4l2-subdev.h>
> >>> +#include <media/videobuf2-core.h>
> >>> +#include <media/videobuf2-v4l2.h>
> >>> +
> >>> +#include "mtk_cam-ipi.h"
> >>> +
> >>> +#define IMG_MAX_WIDTH5376
> >>> +#define IMG_MAX_HEIGHT4032
> >>> +#define IMG_MIN_WIDTH80
> >>> +#define IMG_MIN_HEIGHT60
> >>> +
> >>> +/*
> >>> + * ID enum value for struct mtk_cam_dev_node_desc:id
> >>> + * or mtk_cam_video_device:id
> >>> + */
> >>> +enum  {
> >>> +MTK_CAM_P1_META_IN_0 = 0,
> >>> +MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +MTK_CAM_P1_META_OUT_0,
> >>> +MTK_CAM_P1_META_OUT_1,
> >>> +MTK_CAM_P1_META_OUT_2,
> >>> +MTK_CAM_P1_META_OUT_3,
> >>> +MTK_CAM_P1_TOTAL_NODES
> >>> +};
> >>> +
> >>> +/* Supported image format list */
> >>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
> >>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
> >>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
> >>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
> >>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
> >>> +
> >>> +/* Supported bayer pixel order */
> >>> +#define MTK_CAM_RAW_PXL_ID_B0
> >>> +#define MTK_CAM_RAW_PXL_ID_GB1
> >>> +#define MTK_CAM_RAW_PXL_ID_GR2
> >>> +#define MTK_CAM_RAW_PXL_ID_R3
> >>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
> >>> +
> >>> +/*
> >>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> >>> + *
> >>> + * @frame_seq_no: The frame sequence of frame in driver layer.
> >>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> >>> + *
> >>> + */
> >>> +struct mtk_p1_frame_param {
> >>> +unsigned int frame_seq_no;
> >>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> >>> +} __packed;
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_request - MTK camera device request.
> >>> + *
> >>> + * @req: Embedded struct media request.
> >>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> >>> + * @frame_work: work queue entry for frame transmission to SCP.
> >>> + * @list: List entry of the object for @struct mtk_cam_dev:
> >>> + *        pending_job_list or running_job_list.
> >>> + * @timestamp: Start of frame timestamp in ns
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_request {
> >>> +struct media_request req;
> >>> +struct mtk_p1_frame_param frame_params;
> >>> +struct work_struct frame_work;
> >>> +struct list_head list;
> >>> +u64 timestamp;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> >>> + *
> >>> + * @vbb: Embedded struct vb2_v4l2_buffer.
> >>> + * @list: List entry of the object for @struct mtk_cam_video_device:
> >>> + *        buf_list.
> >>> + * @daddr: The DMA address of this buffer.
> >>> + * @scp_addr: The SCP address of this buffer which
> >>> + *            is only supported for meta input node.
> >>> + * @node_id: The vidoe node id which this buffer belongs to.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_buffer {
> >>> +struct vb2_v4l2_buffer vbb;
> >>> +struct list_head list;
> >>> +/* Intenal part */
> >>> +dma_addr_t daddr;
> >>> +dma_addr_t scp_addr;
> >>> +unsigned int node_id;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> >>> + *
> >>> + * @id: id of the node
> >>> + * @name: name of the node
> >>> + * @cap: supported V4L2 capabilities
> >>> + * @buf_type: supported V4L2 buffer type
> >>> + * @dma_port: the dma ports associated to the node
> >>> + * @link_flags: default media link flags
> >>> + * @smem_alloc: using the smem_dev as alloc device or not
> >>> + * @image: true for image node, false for meta node
> >>> + * @num_fmts: the number of supported node formats
> >>> + * @default_fmt_idx: default format of this node
> >>> + * @max_buf_count: maximum VB2 buffer count
> >>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> >>> + * @fmts: supported format
> >>> + * @frmsizes: supported V4L2 frame size number
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_node_desc {
> >>> +u8 id;
> >>> +const char *name;
> >>> +u32 cap;
> >>> +u32 buf_type;
> >>> +u32 dma_port;
> >>> +u32 link_flags;
> >>> +u8 smem_alloc:1;
> >>> +u8 image:1;
> >>> +u8 num_fmts;
> >>> +u8 default_fmt_idx;
> >>> +u8 max_buf_count;
> >>> +const struct v4l2_ioctl_ops *ioctl_ops;
> >>> +const struct v4l2_format *fmts;
> >>> +const struct v4l2_frmsizeenum *frmsizes;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_video_device - Mediatek video device structure
> >>> + *
> >>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> >>> + * @enabled: Indicate the video device is enabled or not
> >>> + * @desc: The node description of video device
> >>> + * @vdev_fmt: The V4L2 format of video device
> >>> + * @vdev_pad: The media pad graph object of video device
> >>> + * @vbq: A videobuf queue of video device
> >>> + * @vdev: The video device instance
> >>> + * @vdev_lock: Serializes vb2 queue and video device operations
> >>> + * @buf_list: List for enqueue buffers
> >>> + * @buf_list_lock: Lock used to protect buffer list.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_video_device {
> >>> +unsigned int id;
> >>> +unsigned int enabled;
> >>> +struct mtk_cam_dev_node_desc desc;
> >>> +struct v4l2_format vdev_fmt;
> >>> +struct media_pad vdev_pad;
> >>> +struct vb2_queue vbq;
> >>> +struct video_device vdev;
> >>> +/* Serializes vb2 queue and video device operations */
> >>> +struct mutex vdev_lock;
> >>> +struct list_head buf_list;
> >>> +/* Lock used to protect buffer list */
> >>> +spinlock_t buf_list_lock;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev - Mediatek camera device structure.
> >>> + *
> >>> + * @dev: Pointer to device.
> >>> + * @smem_pdev: Pointer to shared memory device.
> >>> + * @pipeline: Media pipeline information.
> >>> + * @media_dev: Media device instance.
> >>> + * @subdev: The V4L2 sub-device instance.
> >>> + * @v4l2_dev: The V4L2 device driver instance.
> >>> + * @notifier: The v4l2_device notifier data.
> >>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> >>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> >>> + * @seninf: Pointer to the seninf sub-device.
> >>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> >>> + * @streaming: Indicate the overall streaming status is on or off.
> >>> + * @enabled_dmas: The enabled dma port information when streaming on.
> >>> + * @enabled_count: Number of enabled video nodes
> >>> + * @stream_count: Number of streaming video nodes
> >>> + * @running_job_count: Nunber of running jobs in the HW driver.
> >>> + * @pending_job_list: List to keep the media requests before en-queue into
> >>> + *                    HW driver.
> >>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> >>> + * @running_job_list: List to keep the media requests after en-queue into
> >>> + *                    HW driver.
> >>> + * @running_job_lock: Protect the running_job_list data.
> >>> + * @op_lock: Serializes driver's VB2 callback operations.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev {
> >>> +struct device *dev;
> >>> +struct device *smem_dev;
> >>> +struct media_pipeline pipeline;
> >>> +struct media_device media_dev;
> >>> +struct v4l2_subdev subdev;
> >>> +struct v4l2_device v4l2_dev;
> >>> +struct v4l2_async_notifier notifier;
> >>> +struct media_pad *subdev_pads;
> >>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> >>> +struct v4l2_subdev *seninf;
> >>> +struct v4l2_subdev *sensor;
> >>> +unsigned int streaming;
> >>> +unsigned int enabled_dmas;
> >>> +unsigned int enabled_count;
> >>> +unsigned int stream_count;
> >>> +unsigned int running_job_count;
> >>> +struct list_head pending_job_list;
> >>> +/* Protect the pending_job_list data */
> >>> +spinlock_t pending_job_lock;
> >>> +struct list_head running_job_list;
> >>> +/* Protect the running_job_list data & running_job_count */
> >>> +spinlock_t running_job_lock;
> >>> +/* Serializes driver's VB2 callback operations */
> >>> +struct mutex op_lock;
> >>> +};
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> >>> +   unsigned int frame_seq_no);
> >>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> >>> +  unsigned int frame_seq_no);
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no);
> >>> +
> >>> +#endif /* __MTK_CAM_H__ */
> >>>
> >>
> >> _______________________________________________
> >> Linux-mediatek mailing list
> >> Linux-mediatek@lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> 
> Regards,
> Helen

Thanks for your comment.

Best regards,


Jungo


^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-05-04 12:27               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-05-04 12:27 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport


Hi Helen;

Sorry for late reply.
Please check my feedback & questions below.

On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> On 4/8/20 11:05 PM, Jungo Lin wrote:
> > Hi Helen:
> >
> > Thanks for your comments.
> >
> > On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >> Hello Jungo,
> >>
> >> I was taking a look at this patch (thanks for the work),
> >> I didn't look in deep details, but I have some comments, please see
> >> below. I hope it helps.
> >>
> >> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>> This patch adds the Mediatek ISP P1 HW control device driver.
> >>> It handles the ISP HW configuration, provides interrupt handling and
> >>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> >>> implement standard V4L2 video driver that utilizes V4L2 and media
> >>> framework APIs. It supports one media device, one sub-device and
> >>> several video devices during initialization. Moreover, it also connects
> >>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> >>> co-process via SCP communication to compose ISP registers in the
> >>> firmware.
> >>>
> >>> (The current metadata interface used in meta input and partial
> >>> meta nodes is only a temporary solution to kick off the driver
> >>> development and is not ready to be reviewed yet.)
> >>>
> >>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> >>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> >>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> >>> ---
> >>> Changes from v6:
> >>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >>>  - Correct auto suspend timer value for suspend/resume issue
> >>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >>>  - Fix KE due to no sen-inf sub-device
> >>> ---
> >>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>
> >> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
> >> capture, which for metadata, and what is being used to deal with requests.
> >>
> >> It would make it easier to review imho.
> >>
> >
> > For file structure design, it was reviewed in the previous patch
> > serials.
> > e.g.
> > https://patchwork.kernel.org/patch/10938137/
> > If you think it is better, I will modify it.
> 
> Right, I saw a suggestion to merge two files there.
> 
> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
> So it reflects a bit the topology.
> But it is also up to you to see if it improves organization or not, it is just a suggestion.
> 

Ok, got your point.
We will discuss how to do internally.

[snip]
> >>> +isp_composer_hw_init(p1_dev);
> >>> +
> >>> +p1_dev->enqueued_frame_seq_no = 0;
> >>> +p1_dev->dequeued_frame_seq_no = 0;
> >>> +p1_dev->composed_frame_seq_no = 0;
> >>> +p1_dev->sof_count = 0;
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >>> +
> >>> +isp_composer_hw_deinit(p1_dev);
> >>> +pm_runtime_mark_last_busy(dev);
> >>> +pm_runtime_put_autosuspend(dev);
> >>> +rproc_shutdown(p1_dev->rproc_handle);
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >>> + struct mtk_cam_dev_request *req)
> >>> +{
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >>> +
> >>> +/* Accumulated frame sequence number */
> >>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> >>> +
> >>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >>> +queue_work(p1_dev->composer_wq, &req->frame_work);
> >>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no,
> >>> +cam->running_job_count);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> >>> +       unsigned int dequeued_frame_seq_no)
> >>> +{
> >>> +dma_addr_t base_addr = p1_dev->composer_iova;
> >>> +struct device *dev = p1_dev->dev;
> >>> +struct mtk_cam_dev_request *req;
> >>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> >>> +unsigned int addr_offset;
> >>> +
> >>> +/* Send V4L2_EVENT_FRAME_SYNC event */
> >>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +
> >>> +p1_dev->sof_count += 1;
> >>> +/* Save frame information */
> >>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> >>> +
> >>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +if (req)
> >>> +req->timestamp = ktime_get_boottime_ns();
> >>> +
> >>> +/* Update CQ base address if needed */
> >>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no);
> >>> +return;
> >>> +}
> >>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> >>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> >>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> >>> +{
> >>> +u32 val;
> >>> +
> >>> +dev_err(p1_dev->dev,
> >>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> >>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> >>> +dev_err(p1_dev->dev,
> >>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> >>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> >>
> >> I think if would be better to transfor those into dev_dbg and add a counter
> >> in debugfs.
> >>
> >
> > These error messages are important for debugging.
> > I suggest to keep in dev_err.
> 
> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
> end user not so much, since end users won't know the meaning of those values.
> 
> For end users a "dma failure" message would be enough, then advanced users can enable
> debug messages to see more.

OK.
Got your point.

> >
> > Moreover, could you give more information about debug counter?
> > I don't get your point.
> > Do you suggest to accumulate the total count of DMA errors?
> 
> 
> Yes, you could have a debugfs entry with error counters like:
> 
> cat /debugfs/mtk_isp/dma_err
> 8
> 
> So it is easier if this error happens very frequent or not.
> In the rkisp1 case we added:
> 
> /debugfs/rkisp1/data_loss
> /debugfs/rkisp1/pic_size_error
> /debugfs/rkisp1/mipi_error
> /debugfs/rkisp1/stats_error
> /debugfs/rkisp1/mp_stop_timeout
> /debugfs/rkisp1/sp_stop_timeout
> /debugfs/rkisp1/mp_frame_drop
> /debugfs/rkisp1/sp_frame_drop
> 
> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
> 
> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
> 

Ok, we will follow your suggestion.


[snip]

> >>> +return;
> >>> +
> >>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
> >>> +
> >>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> >>> +struct vb2_buffer *vb;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct mtk_cam_video_device *node;
> >>> +
> >>> +if (!vb2_request_object_is_buffer(obj))
> >>> +continue;
> >>> +vb = container_of(obj, struct vb2_buffer, req_obj);
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_del(&buf->list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> >>> +vb->timestamp = ts_eof;
> >>> +else
> >>> +vb->timestamp = req->timestamp;
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +}
> >>> +
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +return req;
> >>> +}
> >>> +}
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +
> >>> +return NULL;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> >>> +   unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> >>> +list_del(&req->list);
> >>> +break;
> >>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space for frame drop */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> >>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
> >>> + req->frame_params.frame_seq_no);
> >>
> >> maybe a counter in debugfs instead of the warning.
> >>
> >
> > Do you mean to add counter to accumulate the total count of drop frames?
> 
> please see my comment above.
> 

Ok, add this in next patch.

> > Could we add this and also keep this warning message?
> 
> Userspace would still continue to work when this happens, not sure if it is worthy
> adding a warn, I would move it to dev_dbg() instead IMHO.
> 

Ok, revise in next patch.

[snip]
> >>> +
> >>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> >>> +     struct v4l2_pix_format_mplane *mp)
> >>> +{
> >>> +unsigned int bpl, ppl;
> >>
> >> bytes per line and pixels per line right?
> >>
> >
> > Yes.
> >
> >>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >>
> >> wouldn't be easier a get_pixel_bytes() function instead of bits?
> >>
> >
> > Sorry. I didn't get the point.
> > The unit of return value is bits, not bytes.
> > Do you suggest move bpl & ppl calculation into get_pixel_bits() and
> > rename to get_pixel_bytes()?
> 
> Never mind, I misread it.
> 

Ok, we will skip this.

[snip]
> >>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
> >>> +int ret;
> >>> +
> >>> +/* Get sensor format configuration */
> >>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> >>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> >>> +if (ret) {
> >>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +sd_width = sd_fmt.format.width;
> >>> +sd_height = sd_fmt.format.height;
> >>> +sd_code = sd_fmt.format.code;
> >>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> >>> +sd_code);
> >>
> >> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
> >> it should be configured from userspace.
> >>
> >
> > Could you explain why?
> > Moreover, how does configuration from user space?
> 
> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
> 
> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
> 
> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
> can be configured by userspace and part can't (which iirc should't be done in the media API).
> 
> Do you need to use v4l2_device_register_subdev_nodes() ?
> 
> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
> https://patchwork.kernel.org/cover/11463183/
> 
> which would be more appropriated if you don't want userspace to configure the whole pipeline.
> 

The purpose of P1 sub-device is to provide V4L2 event subscribe with
V4L2_EVENT_FRAME_SYNC event for user space. It is the same
implementation as blow link.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853

As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
precisely.

[snip]

> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +unsigned long flags;
> >>> +
> >>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> >>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> >>> +
> >>> +/* added the buffer into the tracking list */
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_add_tail(&buf->list, &node->buf_list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +
> >>> +/* update buffer internal address */
> >>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> >>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> >>
> >> isn't it an issue if userspace queue two buffers for the same video device in the same request?
> >>
> >> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
> >> will be at req->frame_params.dma_bufs[buf->node_id], no?
> >>
> >> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
> >> buffer?
> >>
> >> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
> >>
> >
> > 1. yes, it is a issue if userspace queues two buffers for the same video
> > device with the same request FD.
> >
> > 2. All buffers which are belonged different to different video devices
> > in the request list will be updated to req->frame_params.dma_bufs by
> > buf->node_id.
> >
> > 3. It is not allowed for userspace to queue partial buffers for all
> > enabled video devices. If it happens, it may trigger DMA errors for this
> > request.
> 
> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
> 

For case 1, it is handled in the vb2_queue_or_prepare_buf.
https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

For case 3, I need to correct my previous answer. This behavior should
be OK for outputted DMA. We have frame buffer controller for each
outputted DMAs. So it means the tuning buffer node is mandatory for each
request, other nodes are optional. We will implement this
in .req_validate to check.

> >
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +dma_addr_t addr;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +buf->node_id = node->id;
> >>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> >>> +buf->scp_addr = 0;
> >>> +
> >>> +/* SCP address is only valid for meta input buffer */
> >>> +if (!node->desc.smem_alloc)
> >>> +return 0;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +/* Use coherent address to get iova address */
> >>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> >>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
> >>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> >>> +return -EFAULT;
> >>> +}
> >>> +buf->scp_addr = buf->daddr;
> >>> +buf->daddr = addr;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +if (vb2_plane_size(vb, 0) < size) {
> >>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> >>> +vb2_plane_size(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> >>> +if (vb2_get_plane_payload(vb, 0) != size) {
> >>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> >>> +vb2_get_plane_payload(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +return 0;
> >>> +}
> >>> +
> >>> +v4l2_buf->field = V4L2_FIELD_NONE;
> >>> +vb2_set_plane_payload(vb, 0, size);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +if (!node->desc.smem_alloc)
> >>> +return;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +dma_unmap_page_attrs(dev, buf->daddr,
> >>> +     vb->planes[0].length,
> >>> +     DMA_BIDIRECTIONAL,
> >>> +     DMA_ATTR_SKIP_CPU_SYNC);
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +
> >>> +dev_dbg(cam->dev, "%s\n", __func__);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> >>> +   unsigned int *num_buffers,
> >>> +   unsigned int *num_planes,
> >>> +   unsigned int sizes[],
> >>> +   struct device *alloc_devs[])
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +unsigned int max_buffer_count = node->desc.max_buf_count;
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +/* Check the limitation of buffer size */
> >>> +if (max_buffer_count)
> >>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> >>> +
> >>> +if (node->desc.smem_alloc)
> >>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> >>> +
> >>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> >>> +if (*num_planes) {
> >>> +if (sizes[0] < size || *num_planes != 1)
> >>> +return -EINVAL;
> >>> +} else {
> >>> +*num_planes = 1;
> >>> +sizes[0] = size;
> >>> +}
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> >>> +   struct mtk_cam_video_device *node,
> >>> +   enum vb2_buffer_state state)
> >>> +{
> >>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> >>> +list_del(&buf->list);
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> >>> +       unsigned int count)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!node->enabled) {
> >>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
> >>> +ret = -ENOLINK;
> >>> +goto fail_ret_buf;
> >>> +}
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +/* Start streaming of the whole pipeline now*/
> >>> +if (!cam->pipeline.streaming_count) {
> >>
> >> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
> >>
> >
> > The check is designed to start the media pipeline when we start
> > streaming on the first node. You could refer the detail in below link.
> >
> > https://patchwork.kernel.org/patch/10985819/
> 
> right, ok, this is when enabling streaming from multiple nodes.
> 
> media_pipeline_start() is usually called for every stream that starts.
> 
> So cam->pipeline.streaming_count can reflect the number of streams enabled.
> 
> So maybe you don't need cam->stream_count.
> 

Ok, revise in next patch.

> >
> >
> >>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
> >>> +goto fail_unlock;
> >>> +}
> >>> +mtk_cam_dev_init_stream(cam);
> >>> +ret = mtk_isp_hw_init(cam);
> 
> Would it make sense to move this to s_stream in the ISP's subdevice ?
> 

No, we like to initialize our ISP firmware here when the first video
node is streaming on. It is too late to initialize when all video nodes
are streaming-on.

> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init HW:%d\n", ret);
> >>> +goto fail_stop_pipeline;
> >>> +}
> >>> +}
> >>> +
> >>> +/* Media links are fixed after media_pipeline_start */
> >>> +cam->stream_count++;
> >>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> >>> +cam->enabled_count);
> >>> +if (cam->stream_count < cam->enabled_count) {
> 
> I'm also wondering, since you need to wait for all the enabled video devices
> to start streaming, shouldn't this be done inside a request? So you can enable
> all of them at once?
> 
> Also, like this you wouldn't need to check enabled links to query for enabled video
> nodes, you can just enable the ones in the request.
> 
> make sense?
> 

Sorry, I didn't get your point about this comment.
Which request could we handle this logic?
Do you mean move this logic into mtk_cam_req_queue function?

> >>> +mutex_unlock(&cam->op_lock);
> >>> +return 0;
> >>> +}
> >>> +
> >>> +/* Stream on sub-devices node */
> >>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> >>> +if (ret)
> >>> +goto fail_no_stream;
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_no_stream:
> >>> +cam->stream_count--;
> >>> +fail_stop_pipeline:
> >>> +if (cam->stream_count == 0)
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +fail_unlock:
> >>> +mutex_unlock(&cam->op_lock);
> >>> +fail_ret_buf:
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> >>> +cam->stream_count);
> >>> +/* Check the first node to stream-off */
> >>> +if (cam->stream_count == cam->enabled_count)
> >>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> >>> +
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >>> +cam->stream_count--;
> >>> +if (cam->stream_count) {
> >>> +mutex_unlock(&cam->op_lock);
> >>> +return;
> >>> +}
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +mtk_cam_dev_req_cleanup(cam);
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> >>> +   struct v4l2_capability *cap)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +
> >>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> >>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> >>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> >>> + dev_name(cam->dev));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> >>> +   struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index >= node->desc.num_fmts)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt = node->vdev_fmt.fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> >>> +  struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +struct device *dev = cam->dev;
> >>> +const struct v4l2_format *dev_fmt;
> >>> +struct v4l2_format try_fmt;
> >>> +
> >>> +memset(&try_fmt, 0, sizeof(try_fmt));
> >>> +try_fmt.type = f->type;
> >>> +
> >>> +/* Validate pixelformat */
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> >>> +if (!dev_fmt) {
> >>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> >>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> >>> +}
> >>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> >>> +
> >>> +/* Validate image width & height range */
> >>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> >>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> >>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> >>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> >>> +/* 4 bytes alignment for width */
> >>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> >>> +
> >>> +/* Only support one plane */
> >>> +try_fmt.fmt.pix_mp.num_planes = 1;
> >>> +
> >>> +/* bytesperline & sizeimage calculation */
> >>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> >>> +
> >>> +/* Constant format fields */
> >>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> >>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> >>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> >>> +
> >>> +*f = try_fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (vb2_is_busy(node->vdev.queue)) {
> >>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> >>> +return -EBUSY;
> >>> +}
> >>> +
> >>> +/* Get the valid format */
> >>> +mtk_cam_vidioc_try_fmt(file, fh, f);
> >>> +/* Configure to video device */
> >>> +node->vdev_fmt = *f;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> >>> +  struct v4l2_frmsizeenum *sizes)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> >>> +const struct v4l2_format *dev_fmt;
> >>> +
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> >>> +if (!dev_fmt || sizes->index)
> >>> +return -EINVAL;
> >>> +
> >>> +sizes->type = node->desc.frmsizes->type;
> >>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> >>> +       sizeof(sizes->stepwise));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> >>> +struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> >>> +     struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> >>> +.subscribe_event = mtk_cam_sd_subscribe_event,
> >>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> >>> +.s_stream =  mtk_cam_sd_s_stream,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> >>> +.core = &mtk_cam_subdev_core_ops,
> >>> +.video = &mtk_cam_subdev_video_ops,
> >>> +};
> >>
> >> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
> >> I wonder if pad ops shouldn't be implemented too (to be verified).
> >>
> >
> > Ok, I will investigate this.
> >
> >>> +
> >>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> >>> +.link_setup = mtk_cam_media_link_setup,
> >>> +.link_validate = v4l2_subdev_link_validate,
> >>> +};
> >>> +
> >>> +static const struct vb2_ops mtk_cam_vb2_ops = {
> >>> +.queue_setup = mtk_cam_vb2_queue_setup,
> >>> +.wait_prepare = vb2_ops_wait_prepare,
> >>> +.wait_finish = vb2_ops_wait_finish,
> >>> +.buf_init = mtk_cam_vb2_buf_init,
> >>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
> >>> +.start_streaming = mtk_cam_vb2_start_streaming,
> >>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
> >>> +.buf_queue = mtk_cam_vb2_buf_queue,
> >>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> >>> +.buf_request_complete = mtk_cam_vb2_request_complete,
> >>> +};> +
> >>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> >>> +.unlocked_ioctl = video_ioctl2,
> >>> +.open = v4l2_fh_open,
> >>> +.release = vb2_fop_release,
> >>> +.poll = vb2_fop_poll,
> >>> +.mmap = vb2_fop_mmap,
> >>> +#ifdef CONFIG_COMPAT
> >>> +.compat_ioctl32 = v4l2_compat_ioctl32,
> >>> +#endif
> >>> +};
> >>> +
> >>> +static const struct media_device_ops mtk_cam_media_ops = {
> >>> +.req_alloc = mtk_cam_req_alloc,
> >>> +.req_free = mtk_cam_req_free,
> >>> +.req_validate = vb2_request_validate,
> >>> +.req_queue = mtk_cam_req_queue,
> >>> +};
> >>> +
> >>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> >>> +  struct media_device *media_dev)
> >>> +{
> >>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> >>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +media_dev->dev = cam->dev;
> >>> +strscpy(media_dev->model, dev_driver_string(dev),
> >>> +sizeof(media_dev->model));
> >>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> >>> + "platform:%s", dev_name(dev));
> >>> +media_dev->hw_revision = 0;
> >>> +media_device_init(media_dev);
> >>> +media_dev->ops = &mtk_cam_media_ops;
> >>> +
> >>> +ret = media_device_register(media_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register media device:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +/* Initialize subdev pads */
> >>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
> >>> +sizeof(*cam->subdev_pads),
> >>> +GFP_KERNEL);
> >>> +if (!cam->subdev_pads) {
> >>> +dev_err(dev, "failed to allocate subdev_pads\n");
> >>> +ret = -ENOMEM;
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> >>> +     cam->subdev_pads);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> >>> +for (i = 0; i < num_pads; i++)
> >>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Customize the last one pad as CIO sink pad. */
> >>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int
> >>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> >>> +      struct mtk_cam_video_device *node)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct video_device *vdev = &node->vdev;
> >>> +struct vb2_queue *vbq = &node->vbq;
> >>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> >>> +unsigned int link_flags = node->desc.link_flags;
> >>> +int ret;
> >>> +
> >>> +/* Initialize mtk_cam_video_device */
> >>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> >>> +node->enabled = true;
> >>> +else
> >>> +node->enabled = false;
> >>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> >>> +
> >>> +cam->subdev_pads[node->id].flags = output ?
> >>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Initialize media entities */
> >>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> >>> +
> >>> +/* Initialize vbq */
> >>> +vbq->type = node->desc.buf_type;
> >>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> >>> +vbq->io_modes = VB2_MMAP;
> >>> +else
> >>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> >>> +
> >>> +if (node->desc.smem_alloc) {
> >>> +vbq->bidirectional = 1;
> >>> +vbq->dev = cam->smem_dev;
> >>> +} else {
> >>> +vbq->dev = dev;
> >>> +}
> >>> +vbq->ops = &mtk_cam_vb2_ops;
> >>> +vbq->mem_ops = &vb2_dma_contig_memops;
> >>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> >>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> >>> +if (output)
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> >>> +else
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> >>> +/* No minimum buffers limitation */
> >>> +vbq->min_buffers_needed = 0;
> >>> +vbq->drv_priv = cam;
> >>> +vbq->lock = &node->vdev_lock;
> >>> +vbq->supports_requests = true;
> >>> +vbq->requires_requests = true;
> >>> +
> >>> +ret = vb2_queue_init(vbq);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> >>> +goto fail_media_clean;
> >>> +}
> >>> +
> >>> +/* Initialize vdev */
> >>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> >>> + dev_driver_string(dev), node->desc.name);
> >>> +/* set cap/type/ioctl_ops of the video device */
> >>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> >>> +vdev->ioctl_ops = node->desc.ioctl_ops;
> >>> +vdev->fops = &mtk_cam_v4l2_fops;
> >>> +vdev->release = video_device_release_empty;
> >>> +vdev->lock = &node->vdev_lock;
> >>> +vdev->v4l2_dev = &cam->v4l2_dev;
> >>> +vdev->queue = &node->vbq;
> >>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> >>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> >>> +vdev->entity.ops = NULL;
> >>> +video_set_drvdata(vdev, cam);
> >>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> >>> +
> >>> +/* Initialize miscellaneous variables */
> >>> +mutex_init(&node->vdev_lock);
> >>> +INIT_LIST_HEAD(&node->buf_list);
> >>> +spin_lock_init(&node->buf_list_lock);
> >>> +
> >>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register vde:%d\n", ret);
> >>> +goto fail_vb2_rel;
> >>> +}
> >>> +
> >>> +/* Create link between video node and the subdev pad */
> >>> +if (output) {
> >>> +ret = media_create_pad_link(&vdev->entity, 0,
> >>> +    &cam->subdev.entity,
> >>> +    node->id, link_flags);
> >>> +} else {
> >>> +ret = media_create_pad_link(&cam->subdev.entity,
> >>> +    node->id, &vdev->entity, 0,
> >>> +    link_flags);
> >>> +}
> >>
> >> No need for the curly braces.
> >>
> >
> > Revised in next patch.
> >
> >>> +if (ret)
> >>> +goto fail_vdev_ureg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_ureg:
> >>> +video_unregister_device(vdev);
> >>> +fail_vb2_rel:
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +vb2_queue_release(vbq);
> >>> +fail_media_clean:
> >>> +media_entity_cleanup(&vdev->entity);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void
> >>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> >>> +{
> >>> +video_unregister_device(&node->vdev);
> >>> +vb2_queue_release(&node->vbq);
> >>> +media_entity_cleanup(&node->vdev.entity);
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +/* Set up media device & pads */
> >>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
> >>> +if (ret)
> >>> +return ret;
> >>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> >>> +
> >>> +/* Set up v4l2 device */
> >>> +cam->v4l2_dev.mdev = &cam->media_dev;
> >>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> >>> +
> >>> +/* Initialize subdev */
> >>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> >>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> >>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> >>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> >>> +V4L2_SUBDEV_FL_HAS_EVENTS;
> >>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> >>> + "%s", dev_driver_string(dev));
> >>> +v4l2_set_subdevdata(&cam->subdev, cam);
> >>> +
> >>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
> >>> +goto fail_clean_media_entiy;
> >>> +}
> >>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
> >>> +
> >>> +/* Create video nodes and links */
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> >>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> >>> +
> >>> +node->id = node->desc.id;
> >>> +ret = mtk_cam_video_register_device(cam, node);
> >>> +if (ret)
> >>> +goto fail_vdev_unreg;
> >>> +}
> >>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_unreg:
> >>> +for (i--; i >= 0; i--)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +fail_clean_media_entiy:
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +int i;
> >>> +
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +
> >>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
> >>> +v4l2_device_unregister_subdev(&cam->subdev);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> >>> +      struct v4l2_subdev *sd,
> >>> +      struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +cam->seninf = sd;
> >>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> >>> +struct v4l2_subdev *sd,
> >>> +struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +cam->seninf = NULL;
> >>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!cam->seninf) {
> >>> +dev_err(dev, "No seninf subdev\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> >>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> >>> +    MEDIA_LNK_FL_IMMUTABLE |
> >>> +    MEDIA_LNK_FL_ENABLED);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
> >>> +cam->seninf->entity.name, cam->subdev.entity.name,
> >>> +ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> >>> +.bound = mtk_cam_dev_notifier_bound,
> >>> +.unbind = mtk_cam_dev_notifier_unbind,
> >>> +.complete = mtk_cam_dev_notifier_complete,
> >>> +};
> >>> +
> >>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +v4l2_async_notifier_init(&cam->notifier);
> >>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> >>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> >>
> >> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
> >>
> >> Regards,
> >> Helen
> >>
> >
> > Ok, we will investigate how to do.
> >
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> >>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> >>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +v4l2_async_notifier_unregister(&cam->notifier);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> >>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> >>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> >>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> >>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> >>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};> +
> >>> +static const struct v4l2_format meta_fmts[] = {
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> >>> +.buffersize = 512 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_3A,
> >>> +.buffersize = 1200 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_AF,
> >>> +.buffersize = 640 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
> >>> +.buffersize = 288 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
> >>> +.buffersize = 256,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format stream_out_fmts[] = {
> >>> +/* This is a default image format */
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format bin_out_fmts[] = {
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc output_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_META_IN_0,
> >>> +.name = "meta input",
> >>> +.cap = V4L2_CAP_META_OUTPUT,
> >>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = true,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 0,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc capture_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +.name = "main stream",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_IMGO,
> >>> +.fmts = stream_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +.name = "packed out",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = 0,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_RRZO,
> >>> +.fmts = bin_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_0,
> >>> +.name = "partial meta 0",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AAO | R_FLKO | R_PSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 1,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_1,
> >>> +.name = "partial meta 1",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AFO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 2,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_2,
> >>> +.name = "partial meta 2",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LCSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 3,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_3,
> >>> +.name = "partial meta 3",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LMVO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 4,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +/* The helper to configure the device context */
> >>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +unsigned int node_idx;
> >>> +int i;
> >>> +
> >>> +node_idx = 0;
> >>> +/* Setup the output queue */
> >>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
> >>> +
> >>> +/* Setup the capture queue */
> >>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> >>> +}
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam)
> >>> +{
> >>> +int ret;
> >>> +
> >>> +cam->dev = &pdev->dev;
> >>> +mtk_cam_dev_queue_setup(cam);
> >>> +
> >>> +spin_lock_init(&cam->pending_job_lock);
> >>> +spin_lock_init(&cam->running_job_lock);
> >>> +INIT_LIST_HEAD(&cam->pending_job_list);
> >>> +INIT_LIST_HEAD(&cam->running_job_list);
> >>> +mutex_init(&cam->op_lock);
> >>> +
> >>> +/* v4l2 sub-device registration */
> >>> +ret = mtk_cam_v4l2_register(cam);
> >>> +if (ret)
> >>> +return ret;
> >>> +
> >>> +ret = mtk_cam_v4l2_async_register(cam);
> >>> +if (ret)
> >>> +goto fail_v4l2_unreg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_v4l2_unreg:
> >>> +mutex_destroy(&cam->op_lock);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +mtk_cam_v4l2_async_unregister(cam);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +mutex_destroy(&cam->op_lock);
> >>> +}
> >>> +
> >>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> new file mode 100644
> >>> index 000000000000..0a340a1e65ea
> >>> --- /dev/null
> >>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> @@ -0,0 +1,244 @@
> >>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>> +/*
> >>> + * Copyright (c) 2019 MediaTek Inc.
> >>> + */
> >>> +
> >>> +#ifndef __MTK_CAM_H__
> >>> +#define __MTK_CAM_H__
> >>> +
> >>> +#include <linux/device.h>
> >>> +#include <linux/types.h>
> >>> +#include <linux/platform_device.h>
> >>> +#include <linux/spinlock.h>
> >>> +#include <linux/videodev2.h>
> >>> +#include <media/v4l2-device.h>
> >>> +#include <media/v4l2-ctrls.h>
> >>> +#include <media/v4l2-subdev.h>
> >>> +#include <media/videobuf2-core.h>
> >>> +#include <media/videobuf2-v4l2.h>
> >>> +
> >>> +#include "mtk_cam-ipi.h"
> >>> +
> >>> +#define IMG_MAX_WIDTH5376
> >>> +#define IMG_MAX_HEIGHT4032
> >>> +#define IMG_MIN_WIDTH80
> >>> +#define IMG_MIN_HEIGHT60
> >>> +
> >>> +/*
> >>> + * ID enum value for struct mtk_cam_dev_node_desc:id
> >>> + * or mtk_cam_video_device:id
> >>> + */
> >>> +enum  {
> >>> +MTK_CAM_P1_META_IN_0 = 0,
> >>> +MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +MTK_CAM_P1_META_OUT_0,
> >>> +MTK_CAM_P1_META_OUT_1,
> >>> +MTK_CAM_P1_META_OUT_2,
> >>> +MTK_CAM_P1_META_OUT_3,
> >>> +MTK_CAM_P1_TOTAL_NODES
> >>> +};
> >>> +
> >>> +/* Supported image format list */
> >>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
> >>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
> >>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
> >>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
> >>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
> >>> +
> >>> +/* Supported bayer pixel order */
> >>> +#define MTK_CAM_RAW_PXL_ID_B0
> >>> +#define MTK_CAM_RAW_PXL_ID_GB1
> >>> +#define MTK_CAM_RAW_PXL_ID_GR2
> >>> +#define MTK_CAM_RAW_PXL_ID_R3
> >>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
> >>> +
> >>> +/*
> >>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> >>> + *
> >>> + * @frame_seq_no: The frame sequence of frame in driver layer.
> >>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> >>> + *
> >>> + */
> >>> +struct mtk_p1_frame_param {
> >>> +unsigned int frame_seq_no;
> >>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> >>> +} __packed;
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_request - MTK camera device request.
> >>> + *
> >>> + * @req: Embedded struct media request.
> >>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> >>> + * @frame_work: work queue entry for frame transmission to SCP.
> >>> + * @list: List entry of the object for @struct mtk_cam_dev:
> >>> + *        pending_job_list or running_job_list.
> >>> + * @timestamp: Start of frame timestamp in ns
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_request {
> >>> +struct media_request req;
> >>> +struct mtk_p1_frame_param frame_params;
> >>> +struct work_struct frame_work;
> >>> +struct list_head list;
> >>> +u64 timestamp;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> >>> + *
> >>> + * @vbb: Embedded struct vb2_v4l2_buffer.
> >>> + * @list: List entry of the object for @struct mtk_cam_video_device:
> >>> + *        buf_list.
> >>> + * @daddr: The DMA address of this buffer.
> >>> + * @scp_addr: The SCP address of this buffer which
> >>> + *            is only supported for meta input node.
> >>> + * @node_id: The vidoe node id which this buffer belongs to.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_buffer {
> >>> +struct vb2_v4l2_buffer vbb;
> >>> +struct list_head list;
> >>> +/* Intenal part */
> >>> +dma_addr_t daddr;
> >>> +dma_addr_t scp_addr;
> >>> +unsigned int node_id;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> >>> + *
> >>> + * @id: id of the node
> >>> + * @name: name of the node
> >>> + * @cap: supported V4L2 capabilities
> >>> + * @buf_type: supported V4L2 buffer type
> >>> + * @dma_port: the dma ports associated to the node
> >>> + * @link_flags: default media link flags
> >>> + * @smem_alloc: using the smem_dev as alloc device or not
> >>> + * @image: true for image node, false for meta node
> >>> + * @num_fmts: the number of supported node formats
> >>> + * @default_fmt_idx: default format of this node
> >>> + * @max_buf_count: maximum VB2 buffer count
> >>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> >>> + * @fmts: supported format
> >>> + * @frmsizes: supported V4L2 frame size number
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_node_desc {
> >>> +u8 id;
> >>> +const char *name;
> >>> +u32 cap;
> >>> +u32 buf_type;
> >>> +u32 dma_port;
> >>> +u32 link_flags;
> >>> +u8 smem_alloc:1;
> >>> +u8 image:1;
> >>> +u8 num_fmts;
> >>> +u8 default_fmt_idx;
> >>> +u8 max_buf_count;
> >>> +const struct v4l2_ioctl_ops *ioctl_ops;
> >>> +const struct v4l2_format *fmts;
> >>> +const struct v4l2_frmsizeenum *frmsizes;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_video_device - Mediatek video device structure
> >>> + *
> >>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> >>> + * @enabled: Indicate the video device is enabled or not
> >>> + * @desc: The node description of video device
> >>> + * @vdev_fmt: The V4L2 format of video device
> >>> + * @vdev_pad: The media pad graph object of video device
> >>> + * @vbq: A videobuf queue of video device
> >>> + * @vdev: The video device instance
> >>> + * @vdev_lock: Serializes vb2 queue and video device operations
> >>> + * @buf_list: List for enqueue buffers
> >>> + * @buf_list_lock: Lock used to protect buffer list.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_video_device {
> >>> +unsigned int id;
> >>> +unsigned int enabled;
> >>> +struct mtk_cam_dev_node_desc desc;
> >>> +struct v4l2_format vdev_fmt;
> >>> +struct media_pad vdev_pad;
> >>> +struct vb2_queue vbq;
> >>> +struct video_device vdev;
> >>> +/* Serializes vb2 queue and video device operations */
> >>> +struct mutex vdev_lock;
> >>> +struct list_head buf_list;
> >>> +/* Lock used to protect buffer list */
> >>> +spinlock_t buf_list_lock;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev - Mediatek camera device structure.
> >>> + *
> >>> + * @dev: Pointer to device.
> >>> + * @smem_pdev: Pointer to shared memory device.
> >>> + * @pipeline: Media pipeline information.
> >>> + * @media_dev: Media device instance.
> >>> + * @subdev: The V4L2 sub-device instance.
> >>> + * @v4l2_dev: The V4L2 device driver instance.
> >>> + * @notifier: The v4l2_device notifier data.
> >>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> >>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> >>> + * @seninf: Pointer to the seninf sub-device.
> >>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> >>> + * @streaming: Indicate the overall streaming status is on or off.
> >>> + * @enabled_dmas: The enabled dma port information when streaming on.
> >>> + * @enabled_count: Number of enabled video nodes
> >>> + * @stream_count: Number of streaming video nodes
> >>> + * @running_job_count: Nunber of running jobs in the HW driver.
> >>> + * @pending_job_list: List to keep the media requests before en-queue into
> >>> + *                    HW driver.
> >>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> >>> + * @running_job_list: List to keep the media requests after en-queue into
> >>> + *                    HW driver.
> >>> + * @running_job_lock: Protect the running_job_list data.
> >>> + * @op_lock: Serializes driver's VB2 callback operations.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev {
> >>> +struct device *dev;
> >>> +struct device *smem_dev;
> >>> +struct media_pipeline pipeline;
> >>> +struct media_device media_dev;
> >>> +struct v4l2_subdev subdev;
> >>> +struct v4l2_device v4l2_dev;
> >>> +struct v4l2_async_notifier notifier;
> >>> +struct media_pad *subdev_pads;
> >>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> >>> +struct v4l2_subdev *seninf;
> >>> +struct v4l2_subdev *sensor;
> >>> +unsigned int streaming;
> >>> +unsigned int enabled_dmas;
> >>> +unsigned int enabled_count;
> >>> +unsigned int stream_count;
> >>> +unsigned int running_job_count;
> >>> +struct list_head pending_job_list;
> >>> +/* Protect the pending_job_list data */
> >>> +spinlock_t pending_job_lock;
> >>> +struct list_head running_job_list;
> >>> +/* Protect the running_job_list data & running_job_count */
> >>> +spinlock_t running_job_lock;
> >>> +/* Serializes driver's VB2 callback operations */
> >>> +struct mutex op_lock;
> >>> +};
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> >>> +   unsigned int frame_seq_no);
> >>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> >>> +  unsigned int frame_seq_no);
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no);
> >>> +
> >>> +#endif /* __MTK_CAM_H__ */
> >>>
> >>
> >> _______________________________________________
> >> Linux-mediatek mailing list
> >> Linux-mediatek@lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> 
> Regards,
> Helen

Thanks for your comment.

Best regards,


Jungo

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-05-04 12:27               ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-05-04 12:27 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport


Hi Helen;

Sorry for late reply.
Please check my feedback & questions below.

On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> On 4/8/20 11:05 PM, Jungo Lin wrote:
> > Hi Helen:
> >
> > Thanks for your comments.
> >
> > On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >> Hello Jungo,
> >>
> >> I was taking a look at this patch (thanks for the work),
> >> I didn't look in deep details, but I have some comments, please see
> >> below. I hope it helps.
> >>
> >> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>> This patch adds the Mediatek ISP P1 HW control device driver.
> >>> It handles the ISP HW configuration, provides interrupt handling and
> >>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
> >>> implement standard V4L2 video driver that utilizes V4L2 and media
> >>> framework APIs. It supports one media device, one sub-device and
> >>> several video devices during initialization. Moreover, it also connects
> >>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
> >>> co-process via SCP communication to compose ISP registers in the
> >>> firmware.
> >>>
> >>> (The current metadata interface used in meta input and partial
> >>> meta nodes is only a temporary solution to kick off the driver
> >>> development and is not ready to be reviewed yet.)
> >>>
> >>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
> >>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
> >>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> >>> ---
> >>> Changes from v6:
> >>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
> >>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
> >>>  - Correct auto suspend timer value for suspend/resume issue
> >>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
> >>>  - Fix KE due to no sen-inf sub-device
> >>> ---
> >>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>
> >> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
> >> capture, which for metadata, and what is being used to deal with requests.
> >>
> >> It would make it easier to review imho.
> >>
> >
> > For file structure design, it was reviewed in the previous patch
> > serials.
> > e.g.
> > https://patchwork.kernel.org/patch/10938137/
> > If you think it is better, I will modify it.
> 
> Right, I saw a suggestion to merge two files there.
> 
> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
> So it reflects a bit the topology.
> But it is also up to you to see if it improves organization or not, it is just a suggestion.
> 

Ok, got your point.
We will discuss how to do internally.

[snip]
> >>> +isp_composer_hw_init(p1_dev);
> >>> +
> >>> +p1_dev->enqueued_frame_seq_no = 0;
> >>> +p1_dev->dequeued_frame_seq_no = 0;
> >>> +p1_dev->composed_frame_seq_no = 0;
> >>> +p1_dev->sof_count = 0;
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
> >>> +
> >>> +isp_composer_hw_deinit(p1_dev);
> >>> +pm_runtime_mark_last_busy(dev);
> >>> +pm_runtime_put_autosuspend(dev);
> >>> +rproc_shutdown(p1_dev->rproc_handle);
> >>> +
> >>> +dev_dbg(dev, "%s done\n", __func__);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
> >>> + struct mtk_cam_dev_request *req)
> >>> +{
> >>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
> >>> +
> >>> +/* Accumulated frame sequence number */
> >>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
> >>> +
> >>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
> >>> +queue_work(p1_dev->composer_wq, &req->frame_work);
> >>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no,
> >>> +cam->running_job_count);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
> >>> +       unsigned int dequeued_frame_seq_no)
> >>> +{
> >>> +dma_addr_t base_addr = p1_dev->composer_iova;
> >>> +struct device *dev = p1_dev->dev;
> >>> +struct mtk_cam_dev_request *req;
> >>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
> >>> +unsigned int addr_offset;
> >>> +
> >>> +/* Send V4L2_EVENT_FRAME_SYNC event */
> >>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +
> >>> +p1_dev->sof_count += 1;
> >>> +/* Save frame information */
> >>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
> >>> +
> >>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
> >>> +if (req)
> >>> +req->timestamp = ktime_get_boottime_ns();
> >>> +
> >>> +/* Update CQ base address if needed */
> >>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no);
> >>> +return;
> >>> +}
> >>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
> >>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
> >>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
> >>> +dev_dbg(dev,
> >>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
> >>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
> >>> +}
> >>> +
> >>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
> >>> +{
> >>> +u32 val;
> >>> +
> >>> +dev_err(p1_dev->dev,
> >>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
> >>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
> >>> +dev_err(p1_dev->dev,
> >>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
> >>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
> >>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
> >>
> >> I think if would be better to transfor those into dev_dbg and add a counter
> >> in debugfs.
> >>
> >
> > These error messages are important for debugging.
> > I suggest to keep in dev_err.
> 
> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
> end user not so much, since end users won't know the meaning of those values.
> 
> For end users a "dma failure" message would be enough, then advanced users can enable
> debug messages to see more.

OK.
Got your point.

> >
> > Moreover, could you give more information about debug counter?
> > I don't get your point.
> > Do you suggest to accumulate the total count of DMA errors?
> 
> 
> Yes, you could have a debugfs entry with error counters like:
> 
> cat /debugfs/mtk_isp/dma_err
> 8
> 
> So it is easier if this error happens very frequent or not.
> In the rkisp1 case we added:
> 
> /debugfs/rkisp1/data_loss
> /debugfs/rkisp1/pic_size_error
> /debugfs/rkisp1/mipi_error
> /debugfs/rkisp1/stats_error
> /debugfs/rkisp1/mp_stop_timeout
> /debugfs/rkisp1/sp_stop_timeout
> /debugfs/rkisp1/mp_frame_drop
> /debugfs/rkisp1/sp_frame_drop
> 
> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
> 
> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
> 

Ok, we will follow your suggestion.


[snip]

> >>> +return;
> >>> +
> >>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
> >>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
> >>> +
> >>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
> >>> +struct vb2_buffer *vb;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct mtk_cam_video_device *node;
> >>> +
> >>> +if (!vb2_request_object_is_buffer(obj))
> >>> +continue;
> >>> +vb = container_of(obj, struct vb2_buffer, req_obj);
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_del(&buf->list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> >>> +vb->timestamp = ts_eof;
> >>> +else
> >>> +vb->timestamp = req->timestamp;
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +}
> >>> +
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +return req;
> >>> +}
> >>> +}
> >>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
> >>> +
> >>> +return NULL;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
> >>> +   unsigned int frame_seq_no)
> >>> +{
> >>> +struct mtk_cam_dev_request *req, *req_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&cam->running_job_lock, flags);
> >>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
> >>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
> >>> +req->frame_params.frame_seq_no, frame_seq_no);
> >>> +
> >>> +/* Match by the en-queued request number */
> >>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
> >>> +list_del(&req->list);
> >>> +break;
> >>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
> >>> +cam->running_job_count--;
> >>> +/* Pass to user space for frame drop */
> >>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
> >>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
> >>> + req->frame_params.frame_seq_no);
> >>
> >> maybe a counter in debugfs instead of the warning.
> >>
> >
> > Do you mean to add counter to accumulate the total count of drop frames?
> 
> please see my comment above.
> 

Ok, add this in next patch.

> > Could we add this and also keep this warning message?
> 
> Userspace would still continue to work when this happens, not sure if it is worthy
> adding a warn, I would move it to dev_dbg() instead IMHO.
> 

Ok, revise in next patch.

[snip]
> >>> +
> >>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
> >>> +     struct v4l2_pix_format_mplane *mp)
> >>> +{
> >>> +unsigned int bpl, ppl;
> >>
> >> bytes per line and pixels per line right?
> >>
> >
> > Yes.
> >
> >>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
> >>
> >> wouldn't be easier a get_pixel_bytes() function instead of bits?
> >>
> >
> > Sorry. I didn't get the point.
> > The unit of return value is bits, not bytes.
> > Do you suggest move bpl & ppl calculation into get_pixel_bits() and
> > rename to get_pixel_bytes()?
> 
> Never mind, I misread it.
> 

Ok, we will skip this.

[snip]
> >>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
> >>> +int ret;
> >>> +
> >>> +/* Get sensor format configuration */
> >>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> >>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
> >>> +if (ret) {
> >>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +sd_width = sd_fmt.format.width;
> >>> +sd_height = sd_fmt.format.height;
> >>> +sd_code = sd_fmt.format.code;
> >>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
> >>> +sd_code);
> >>
> >> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
> >> it should be configured from userspace.
> >>
> >
> > Could you explain why?
> > Moreover, how does configuration from user space?
> 
> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
> 
> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
> 
> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
> can be configured by userspace and part can't (which iirc should't be done in the media API).
> 
> Do you need to use v4l2_device_register_subdev_nodes() ?
> 
> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
> https://patchwork.kernel.org/cover/11463183/
> 
> which would be more appropriated if you don't want userspace to configure the whole pipeline.
> 

The purpose of P1 sub-device is to provide V4L2 event subscribe with
V4L2_EVENT_FRAME_SYNC event for user space. It is the same
implementation as blow link.
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853

As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
precisely.

[snip]

> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +unsigned long flags;
> >>> +
> >>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
> >>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
> >>> +
> >>> +/* added the buffer into the tracking list */
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_add_tail(&buf->list, &node->buf_list);
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +
> >>> +/* update buffer internal address */
> >>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
> >>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
> >>
> >> isn't it an issue if userspace queue two buffers for the same video device in the same request?
> >>
> >> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
> >> will be at req->frame_params.dma_bufs[buf->node_id], no?
> >>
> >> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
> >> buffer?
> >>
> >> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
> >>
> >
> > 1. yes, it is a issue if userspace queues two buffers for the same video
> > device with the same request FD.
> >
> > 2. All buffers which are belonged different to different video devices
> > in the request list will be updated to req->frame_params.dma_bufs by
> > buf->node_id.
> >
> > 3. It is not allowed for userspace to queue partial buffers for all
> > enabled video devices. If it happens, it may trigger DMA errors for this
> > request.
> 
> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
> 

For case 1, it is handled in the vb2_queue_or_prepare_buf.
https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

For case 3, I need to correct my previous answer. This behavior should
be OK for outputted DMA. We have frame buffer controller for each
outputted DMAs. So it means the tuning buffer node is mandatory for each
request, other nodes are optional. We will implement this
in .req_validate to check.

> >
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct device *dev = cam->dev;
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +dma_addr_t addr;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +buf->node_id = node->id;
> >>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
> >>> +buf->scp_addr = 0;
> >>> +
> >>> +/* SCP address is only valid for meta input buffer */
> >>> +if (!node->desc.smem_alloc)
> >>> +return 0;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +/* Use coherent address to get iova address */
> >>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
> >>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
> >>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
> >>> +return -EFAULT;
> >>> +}
> >>> +buf->scp_addr = buf->daddr;
> >>> +buf->daddr = addr;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +if (vb2_plane_size(vb, 0) < size) {
> >>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
> >>> +vb2_plane_size(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +
> >>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> >>> +if (vb2_get_plane_payload(vb, 0) != size) {
> >>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
> >>> +vb2_get_plane_payload(vb, 0), size);
> >>> +return -EINVAL;
> >>> +}
> >>> +return 0;
> >>> +}
> >>> +
> >>> +v4l2_buf->field = V4L2_FIELD_NONE;
> >>> +vb2_set_plane_payload(vb, 0, size);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +struct mtk_cam_dev_buffer *buf;
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +if (!node->desc.smem_alloc)
> >>> +return;
> >>> +
> >>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
> >>> +dma_unmap_page_attrs(dev, buf->daddr,
> >>> +     vb->planes[0].length,
> >>> +     DMA_BIDIRECTIONAL,
> >>> +     DMA_ATTR_SKIP_CPU_SYNC);
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
> >>> +
> >>> +dev_dbg(cam->dev, "%s\n", __func__);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> >>> +   unsigned int *num_buffers,
> >>> +   unsigned int *num_planes,
> >>> +   unsigned int sizes[],
> >>> +   struct device *alloc_devs[])
> >>> +{
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +unsigned int max_buffer_count = node->desc.max_buf_count;
> >>> +const struct v4l2_format *fmt = &node->vdev_fmt;
> >>> +unsigned int size;
> >>> +
> >>> +/* Check the limitation of buffer size */
> >>> +if (max_buffer_count)
> >>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> >>> +
> >>> +if (node->desc.smem_alloc)
> >>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> >>> +
> >>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> >>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> >>> +size = fmt->fmt.meta.buffersize;
> >>> +else
> >>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> >>> +
> >>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> >>> +if (*num_planes) {
> >>> +if (sizes[0] < size || *num_planes != 1)
> >>> +return -EINVAL;
> >>> +} else {
> >>> +*num_planes = 1;
> >>> +sizes[0] = size;
> >>> +}
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
> >>> +   struct mtk_cam_video_device *node,
> >>> +   enum vb2_buffer_state state)
> >>> +{
> >>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
> >>> +unsigned long flags;
> >>> +
> >>> +spin_lock_irqsave(&node->buf_list_lock, flags);
> >>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
> >>> +list_del(&buf->list);
> >>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
> >>> +}
> >>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
> >>> +       unsigned int count)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!node->enabled) {
> >>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
> >>> +ret = -ENOLINK;
> >>> +goto fail_ret_buf;
> >>> +}
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +/* Start streaming of the whole pipeline now*/
> >>> +if (!cam->pipeline.streaming_count) {
> >>
> >> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
> >>
> >
> > The check is designed to start the media pipeline when we start
> > streaming on the first node. You could refer the detail in below link.
> >
> > https://patchwork.kernel.org/patch/10985819/
> 
> right, ok, this is when enabling streaming from multiple nodes.
> 
> media_pipeline_start() is usually called for every stream that starts.
> 
> So cam->pipeline.streaming_count can reflect the number of streams enabled.
> 
> So maybe you don't need cam->stream_count.
> 

Ok, revise in next patch.

> >
> >
> >>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
> >>> +goto fail_unlock;
> >>> +}
> >>> +mtk_cam_dev_init_stream(cam);
> >>> +ret = mtk_isp_hw_init(cam);
> 
> Would it make sense to move this to s_stream in the ISP's subdevice ?
> 

No, we like to initialize our ISP firmware here when the first video
node is streaming on. It is too late to initialize when all video nodes
are streaming-on.

> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init HW:%d\n", ret);
> >>> +goto fail_stop_pipeline;
> >>> +}
> >>> +}
> >>> +
> >>> +/* Media links are fixed after media_pipeline_start */
> >>> +cam->stream_count++;
> >>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
> >>> +cam->enabled_count);
> >>> +if (cam->stream_count < cam->enabled_count) {
> 
> I'm also wondering, since you need to wait for all the enabled video devices
> to start streaming, shouldn't this be done inside a request? So you can enable
> all of them at once?
> 
> Also, like this you wouldn't need to check enabled links to query for enabled video
> nodes, you can just enable the ones in the request.
> 
> make sense?
> 

Sorry, I didn't get your point about this comment.
Which request could we handle this logic?
Do you mean move this logic into mtk_cam_req_queue function?

> >>> +mutex_unlock(&cam->op_lock);
> >>> +return 0;
> >>> +}
> >>> +
> >>> +/* Stream on sub-devices node */
> >>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
> >>> +if (ret)
> >>> +goto fail_no_stream;
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_no_stream:
> >>> +cam->stream_count--;
> >>> +fail_stop_pipeline:
> >>> +if (cam->stream_count == 0)
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +fail_unlock:
> >>> +mutex_unlock(&cam->op_lock);
> >>> +fail_ret_buf:
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
> >>> +{
> >>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
> >>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> >>> +struct device *dev = cam->dev;
> >>> +
> >>> +mutex_lock(&cam->op_lock);
> >>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
> >>> +cam->stream_count);
> >>> +/* Check the first node to stream-off */
> >>> +if (cam->stream_count == cam->enabled_count)
> >>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
> >>> +
> >>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
> >>> +cam->stream_count--;
> >>> +if (cam->stream_count) {
> >>> +mutex_unlock(&cam->op_lock);
> >>> +return;
> >>> +}
> >>> +mutex_unlock(&cam->op_lock);
> >>> +
> >>> +mtk_cam_dev_req_cleanup(cam);
> >>> +media_pipeline_stop(&node->vdev.entity);
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
> >>> +   struct v4l2_capability *cap)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +
> >>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
> >>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
> >>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> >>> + dev_name(cam->dev));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
> >>> +   struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index >= node->desc.num_fmts)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt = node->vdev_fmt.fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
> >>> +  struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +struct device *dev = cam->dev;
> >>> +const struct v4l2_format *dev_fmt;
> >>> +struct v4l2_format try_fmt;
> >>> +
> >>> +memset(&try_fmt, 0, sizeof(try_fmt));
> >>> +try_fmt.type = f->type;
> >>> +
> >>> +/* Validate pixelformat */
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> >>> +if (!dev_fmt) {
> >>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
> >>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
> >>> +}
> >>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> >>> +
> >>> +/* Validate image width & height range */
> >>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> >>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> >>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> >>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> >>> +/* 4 bytes alignment for width */
> >>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
> >>> +
> >>> +/* Only support one plane */
> >>> +try_fmt.fmt.pix_mp.num_planes = 1;
> >>> +
> >>> +/* bytesperline & sizeimage calculation */
> >>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
> >>> +
> >>> +/* Constant format fields */
> >>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> >>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> >>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> >>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> >>> +
> >>> +*f = try_fmt;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
> >>> +struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_dev *cam = video_drvdata(file);
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (vb2_is_busy(node->vdev.queue)) {
> >>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
> >>> +return -EBUSY;
> >>> +}
> >>> +
> >>> +/* Get the valid format */
> >>> +mtk_cam_vidioc_try_fmt(file, fh, f);
> >>> +/* Configure to video device */
> >>> +node->vdev_fmt = *f;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
> >>> +  struct v4l2_frmsizeenum *sizes)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
> >>> +const struct v4l2_format *dev_fmt;
> >>> +
> >>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
> >>> +if (!dev_fmt || sizes->index)
> >>> +return -EINVAL;
> >>> +
> >>> +sizes->type = node->desc.frmsizes->type;
> >>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
> >>> +       sizeof(sizes->stepwise));
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
> >>> +struct v4l2_fmtdesc *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +if (f->index)
> >>> +return -EINVAL;
> >>> +
> >>> +/* f->description is filled in v4l_fill_fmtdesc function */
> >>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->flags = 0;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
> >>> +     struct v4l2_format *f)
> >>> +{
> >>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
> >>> +
> >>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
> >>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
> >>> +.subscribe_event = mtk_cam_sd_subscribe_event,
> >>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
> >>> +.s_stream =  mtk_cam_sd_s_stream,
> >>> +};
> >>> +
> >>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
> >>> +.core = &mtk_cam_subdev_core_ops,
> >>> +.video = &mtk_cam_subdev_video_ops,
> >>> +};
> >>
> >> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
> >> I wonder if pad ops shouldn't be implemented too (to be verified).
> >>
> >
> > Ok, I will investigate this.
> >
> >>> +
> >>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
> >>> +.link_setup = mtk_cam_media_link_setup,
> >>> +.link_validate = v4l2_subdev_link_validate,
> >>> +};
> >>> +
> >>> +static const struct vb2_ops mtk_cam_vb2_ops = {
> >>> +.queue_setup = mtk_cam_vb2_queue_setup,
> >>> +.wait_prepare = vb2_ops_wait_prepare,
> >>> +.wait_finish = vb2_ops_wait_finish,
> >>> +.buf_init = mtk_cam_vb2_buf_init,
> >>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
> >>> +.start_streaming = mtk_cam_vb2_start_streaming,
> >>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
> >>> +.buf_queue = mtk_cam_vb2_buf_queue,
> >>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
> >>> +.buf_request_complete = mtk_cam_vb2_request_complete,
> >>> +};> +
> >>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
> >>> +.unlocked_ioctl = video_ioctl2,
> >>> +.open = v4l2_fh_open,
> >>> +.release = vb2_fop_release,
> >>> +.poll = vb2_fop_poll,
> >>> +.mmap = vb2_fop_mmap,
> >>> +#ifdef CONFIG_COMPAT
> >>> +.compat_ioctl32 = v4l2_compat_ioctl32,
> >>> +#endif
> >>> +};
> >>> +
> >>> +static const struct media_device_ops mtk_cam_media_ops = {
> >>> +.req_alloc = mtk_cam_req_alloc,
> >>> +.req_free = mtk_cam_req_free,
> >>> +.req_validate = vb2_request_validate,
> >>> +.req_queue = mtk_cam_req_queue,
> >>> +};
> >>> +
> >>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
> >>> +  struct media_device *media_dev)
> >>> +{
> >>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
> >>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +media_dev->dev = cam->dev;
> >>> +strscpy(media_dev->model, dev_driver_string(dev),
> >>> +sizeof(media_dev->model));
> >>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> >>> + "platform:%s", dev_name(dev));
> >>> +media_dev->hw_revision = 0;
> >>> +media_device_init(media_dev);
> >>> +media_dev->ops = &mtk_cam_media_ops;
> >>> +
> >>> +ret = media_device_register(media_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register media device:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +/* Initialize subdev pads */
> >>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
> >>> +sizeof(*cam->subdev_pads),
> >>> +GFP_KERNEL);
> >>> +if (!cam->subdev_pads) {
> >>> +dev_err(dev, "failed to allocate subdev_pads\n");
> >>> +ret = -ENOMEM;
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
> >>> +     cam->subdev_pads);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +
> >>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
> >>> +for (i = 0; i < num_pads; i++)
> >>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Customize the last one pad as CIO sink pad. */
> >>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int
> >>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
> >>> +      struct mtk_cam_video_device *node)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +struct video_device *vdev = &node->vdev;
> >>> +struct vb2_queue *vbq = &node->vbq;
> >>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
> >>> +unsigned int link_flags = node->desc.link_flags;
> >>> +int ret;
> >>> +
> >>> +/* Initialize mtk_cam_video_device */
> >>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
> >>> +node->enabled = true;
> >>> +else
> >>> +node->enabled = false;
> >>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
> >>> +
> >>> +cam->subdev_pads[node->id].flags = output ?
> >>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> >>> +
> >>> +/* Initialize media entities */
> >>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> >>> +
> >>> +/* Initialize vbq */
> >>> +vbq->type = node->desc.buf_type;
> >>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
> >>> +vbq->io_modes = VB2_MMAP;
> >>> +else
> >>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
> >>> +
> >>> +if (node->desc.smem_alloc) {
> >>> +vbq->bidirectional = 1;
> >>> +vbq->dev = cam->smem_dev;
> >>> +} else {
> >>> +vbq->dev = dev;
> >>> +}
> >>> +vbq->ops = &mtk_cam_vb2_ops;
> >>> +vbq->mem_ops = &vb2_dma_contig_memops;
> >>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
> >>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
> >>> +if (output)
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
> >>> +else
> >>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
> >>> +/* No minimum buffers limitation */
> >>> +vbq->min_buffers_needed = 0;
> >>> +vbq->drv_priv = cam;
> >>> +vbq->lock = &node->vdev_lock;
> >>> +vbq->supports_requests = true;
> >>> +vbq->requires_requests = true;
> >>> +
> >>> +ret = vb2_queue_init(vbq);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
> >>> +goto fail_media_clean;
> >>> +}
> >>> +
> >>> +/* Initialize vdev */
> >>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> >>> + dev_driver_string(dev), node->desc.name);
> >>> +/* set cap/type/ioctl_ops of the video device */
> >>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
> >>> +vdev->ioctl_ops = node->desc.ioctl_ops;
> >>> +vdev->fops = &mtk_cam_v4l2_fops;
> >>> +vdev->release = video_device_release_empty;
> >>> +vdev->lock = &node->vdev_lock;
> >>> +vdev->v4l2_dev = &cam->v4l2_dev;
> >>> +vdev->queue = &node->vbq;
> >>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
> >>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
> >>> +vdev->entity.ops = NULL;
> >>> +video_set_drvdata(vdev, cam);
> >>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
> >>> +
> >>> +/* Initialize miscellaneous variables */
> >>> +mutex_init(&node->vdev_lock);
> >>> +INIT_LIST_HEAD(&node->buf_list);
> >>> +spin_lock_init(&node->buf_list_lock);
> >>> +
> >>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register vde:%d\n", ret);
> >>> +goto fail_vb2_rel;
> >>> +}
> >>> +
> >>> +/* Create link between video node and the subdev pad */
> >>> +if (output) {
> >>> +ret = media_create_pad_link(&vdev->entity, 0,
> >>> +    &cam->subdev.entity,
> >>> +    node->id, link_flags);
> >>> +} else {
> >>> +ret = media_create_pad_link(&cam->subdev.entity,
> >>> +    node->id, &vdev->entity, 0,
> >>> +    link_flags);
> >>> +}
> >>
> >> No need for the curly braces.
> >>
> >
> > Revised in next patch.
> >
> >>> +if (ret)
> >>> +goto fail_vdev_ureg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_ureg:
> >>> +video_unregister_device(vdev);
> >>> +fail_vb2_rel:
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +vb2_queue_release(vbq);
> >>> +fail_media_clean:
> >>> +media_entity_cleanup(&vdev->entity);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void
> >>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
> >>> +{
> >>> +video_unregister_device(&node->vdev);
> >>> +vb2_queue_release(&node->vbq);
> >>> +media_entity_cleanup(&node->vdev.entity);
> >>> +mutex_destroy(&node->vdev_lock);
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int i, ret;
> >>> +
> >>> +/* Set up media device & pads */
> >>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
> >>> +if (ret)
> >>> +return ret;
> >>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
> >>> +
> >>> +/* Set up v4l2 device */
> >>> +cam->v4l2_dev.mdev = &cam->media_dev;
> >>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
> >>> +goto fail_media_unreg;
> >>> +}
> >>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
> >>> +
> >>> +/* Initialize subdev */
> >>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
> >>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> >>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
> >>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> >>> +V4L2_SUBDEV_FL_HAS_EVENTS;
> >>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
> >>> + "%s", dev_driver_string(dev));
> >>> +v4l2_set_subdevdata(&cam->subdev, cam);
> >>> +
> >>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
> >>> +goto fail_clean_media_entiy;
> >>> +}
> >>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
> >>> +
> >>> +/* Create video nodes and links */
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
> >>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
> >>> +
> >>> +node->id = node->desc.id;
> >>> +ret = mtk_cam_video_register_device(cam, node);
> >>> +if (ret)
> >>> +goto fail_vdev_unreg;
> >>> +}
> >>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_vdev_unreg:
> >>> +for (i--; i >= 0; i--)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +fail_clean_media_entiy:
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +fail_media_unreg:
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +int i;
> >>> +
> >>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
> >>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
> >>> +
> >>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
> >>> +v4l2_device_unregister_subdev(&cam->subdev);
> >>> +v4l2_device_unregister(&cam->v4l2_dev);
> >>> +media_entity_cleanup(&cam->subdev.entity);
> >>> +media_device_unregister(&cam->media_dev);
> >>> +media_device_cleanup(&cam->media_dev);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
> >>> +      struct v4l2_subdev *sd,
> >>> +      struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
> >>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +cam->seninf = sd;
> >>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
> >>> +struct v4l2_subdev *sd,
> >>> +struct v4l2_async_subdev *asd)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +
> >>> +cam->seninf = NULL;
> >>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
> >>> +}
> >>> +
> >>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
> >>> +{
> >>> +struct mtk_cam_dev *cam =
> >>> +container_of(notifier, struct mtk_cam_dev, notifier);
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +if (!cam->seninf) {
> >>> +dev_err(dev, "No seninf subdev\n");
> >>> +return -ENODEV;
> >>> +}
> >>> +
> >>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
> >>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
> >>> +    MEDIA_LNK_FL_IMMUTABLE |
> >>> +    MEDIA_LNK_FL_ENABLED);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
> >>> +cam->seninf->entity.name, cam->subdev.entity.name,
> >>> +ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
> >>> +.bound = mtk_cam_dev_notifier_bound,
> >>> +.unbind = mtk_cam_dev_notifier_unbind,
> >>> +.complete = mtk_cam_dev_notifier_complete,
> >>> +};
> >>> +
> >>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
> >>> +{
> >>> +struct device *dev = cam->dev;
> >>> +int ret;
> >>> +
> >>> +v4l2_async_notifier_init(&cam->notifier);
> >>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
> >>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
> >>
> >> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
> >>
> >> Regards,
> >> Helen
> >>
> >
> > Ok, we will investigate how to do.
> >
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
> >>> +return ret;
> >>> +}
> >>> +
> >>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
> >>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
> >>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
> >>> +if (ret) {
> >>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
> >>> +{
> >>> +v4l2_async_notifier_unregister(&cam->notifier);
> >>> +v4l2_async_notifier_cleanup(&cam->notifier);
> >>> +}
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
> >>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
> >>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
> >>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
> >>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> >>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};
> >>> +
> >>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
> >>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
> >>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
> >>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
> >>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
> >>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
> >>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >>> +.vidioc_querybuf = vb2_ioctl_querybuf,
> >>> +.vidioc_qbuf = vb2_ioctl_qbuf,
> >>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
> >>> +.vidioc_streamon = vb2_ioctl_streamon,
> >>> +.vidioc_streamoff = vb2_ioctl_streamoff,
> >>> +.vidioc_expbuf = vb2_ioctl_expbuf,
> >>> +};> +
> >>> +static const struct v4l2_format meta_fmts[] = {
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
> >>> +.buffersize = 512 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_3A,
> >>> +.buffersize = 1200 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_AF,
> >>> +.buffersize = 640 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
> >>> +.buffersize = 288 * SZ_1K,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.meta = {
> >>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
> >>> +.buffersize = 256,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format stream_out_fmts[] = {
> >>> +/* This is a default image format */
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct v4l2_format bin_out_fmts[] = {
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
> >>> +},
> >>> +},
> >>> +{
> >>> +.fmt.pix_mp = {
> >>> +.width = IMG_MAX_WIDTH,
> >>> +.height = IMG_MAX_HEIGHT,
> >>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
> >>> +},
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc output_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_META_IN_0,
> >>> +.name = "meta input",
> >>> +.cap = V4L2_CAP_META_OUTPUT,
> >>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = true,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 0,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +static const struct
> >>> +mtk_cam_dev_node_desc capture_queues[] = {
> >>> +{
> >>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +.name = "main stream",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_IMGO,
> >>> +.fmts = stream_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +.name = "packed out",
> >>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
> >>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> >>> +.link_flags = 0,
> >>> +.image = true,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_RRZO,
> >>> +.fmts = bin_out_fmts,
> >>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
> >>> +.default_fmt_idx = 0,
> >>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
> >>> +.frmsizes = &(struct v4l2_frmsizeenum) {
> >>> +.index = 0,
> >>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> >>> +.stepwise = {
> >>> +.max_width = IMG_MAX_WIDTH,
> >>> +.min_width = IMG_MIN_WIDTH,
> >>> +.max_height = IMG_MAX_HEIGHT,
> >>> +.min_height = IMG_MIN_HEIGHT,
> >>> +.step_height = 1,
> >>> +.step_width = 1,
> >>> +},
> >>> +},
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_0,
> >>> +.name = "partial meta 0",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AAO | R_FLKO | R_PSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 1,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_1,
> >>> +.name = "partial meta 1",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_AFO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 2,
> >>> +.max_buf_count = 5,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_2,
> >>> +.name = "partial meta 2",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LCSO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 3,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +{
> >>> +.id = MTK_CAM_P1_META_OUT_3,
> >>> +.name = "partial meta 3",
> >>> +.cap = V4L2_CAP_META_CAPTURE,
> >>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
> >>> +.link_flags = 0,
> >>> +.image = false,
> >>> +.smem_alloc = false,
> >>> +.dma_port = R_LMVO,
> >>> +.fmts = meta_fmts,
> >>> +.default_fmt_idx = 4,
> >>> +.max_buf_count = 10,
> >>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
> >>> +},
> >>> +};
> >>> +
> >>> +/* The helper to configure the device context */
> >>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +unsigned int node_idx;
> >>> +int i;
> >>> +
> >>> +node_idx = 0;
> >>> +/* Setup the output queue */
> >>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
> >>> +
> >>> +/* Setup the capture queue */
> >>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
> >>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
> >>> +}
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam)
> >>> +{
> >>> +int ret;
> >>> +
> >>> +cam->dev = &pdev->dev;
> >>> +mtk_cam_dev_queue_setup(cam);
> >>> +
> >>> +spin_lock_init(&cam->pending_job_lock);
> >>> +spin_lock_init(&cam->running_job_lock);
> >>> +INIT_LIST_HEAD(&cam->pending_job_list);
> >>> +INIT_LIST_HEAD(&cam->running_job_list);
> >>> +mutex_init(&cam->op_lock);
> >>> +
> >>> +/* v4l2 sub-device registration */
> >>> +ret = mtk_cam_v4l2_register(cam);
> >>> +if (ret)
> >>> +return ret;
> >>> +
> >>> +ret = mtk_cam_v4l2_async_register(cam);
> >>> +if (ret)
> >>> +goto fail_v4l2_unreg;
> >>> +
> >>> +return 0;
> >>> +
> >>> +fail_v4l2_unreg:
> >>> +mutex_destroy(&cam->op_lock);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +
> >>> +return ret;
> >>> +}
> >>> +
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
> >>> +{
> >>> +mtk_cam_v4l2_async_unregister(cam);
> >>> +mtk_cam_v4l2_unregister(cam);
> >>> +mutex_destroy(&cam->op_lock);
> >>> +}
> >>> +
> >>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> new file mode 100644
> >>> index 000000000000..0a340a1e65ea
> >>> --- /dev/null
> >>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>> @@ -0,0 +1,244 @@
> >>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>> +/*
> >>> + * Copyright (c) 2019 MediaTek Inc.
> >>> + */
> >>> +
> >>> +#ifndef __MTK_CAM_H__
> >>> +#define __MTK_CAM_H__
> >>> +
> >>> +#include <linux/device.h>
> >>> +#include <linux/types.h>
> >>> +#include <linux/platform_device.h>
> >>> +#include <linux/spinlock.h>
> >>> +#include <linux/videodev2.h>
> >>> +#include <media/v4l2-device.h>
> >>> +#include <media/v4l2-ctrls.h>
> >>> +#include <media/v4l2-subdev.h>
> >>> +#include <media/videobuf2-core.h>
> >>> +#include <media/videobuf2-v4l2.h>
> >>> +
> >>> +#include "mtk_cam-ipi.h"
> >>> +
> >>> +#define IMG_MAX_WIDTH5376
> >>> +#define IMG_MAX_HEIGHT4032
> >>> +#define IMG_MIN_WIDTH80
> >>> +#define IMG_MIN_HEIGHT60
> >>> +
> >>> +/*
> >>> + * ID enum value for struct mtk_cam_dev_node_desc:id
> >>> + * or mtk_cam_video_device:id
> >>> + */
> >>> +enum  {
> >>> +MTK_CAM_P1_META_IN_0 = 0,
> >>> +MTK_CAM_P1_MAIN_STREAM_OUT,
> >>> +MTK_CAM_P1_PACKED_BIN_OUT,
> >>> +MTK_CAM_P1_META_OUT_0,
> >>> +MTK_CAM_P1_META_OUT_1,
> >>> +MTK_CAM_P1_META_OUT_2,
> >>> +MTK_CAM_P1_META_OUT_3,
> >>> +MTK_CAM_P1_TOTAL_NODES
> >>> +};
> >>> +
> >>> +/* Supported image format list */
> >>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
> >>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
> >>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
> >>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
> >>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
> >>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
> >>> +
> >>> +/* Supported bayer pixel order */
> >>> +#define MTK_CAM_RAW_PXL_ID_B0
> >>> +#define MTK_CAM_RAW_PXL_ID_GB1
> >>> +#define MTK_CAM_RAW_PXL_ID_GR2
> >>> +#define MTK_CAM_RAW_PXL_ID_R3
> >>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
> >>> +
> >>> +/*
> >>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
> >>> + *
> >>> + * @frame_seq_no: The frame sequence of frame in driver layer.
> >>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
> >>> + *
> >>> + */
> >>> +struct mtk_p1_frame_param {
> >>> +unsigned int frame_seq_no;
> >>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
> >>> +} __packed;
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_request - MTK camera device request.
> >>> + *
> >>> + * @req: Embedded struct media request.
> >>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> >>> + * @frame_work: work queue entry for frame transmission to SCP.
> >>> + * @list: List entry of the object for @struct mtk_cam_dev:
> >>> + *        pending_job_list or running_job_list.
> >>> + * @timestamp: Start of frame timestamp in ns
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_request {
> >>> +struct media_request req;
> >>> +struct mtk_p1_frame_param frame_params;
> >>> +struct work_struct frame_work;
> >>> +struct list_head list;
> >>> +u64 timestamp;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
> >>> + *
> >>> + * @vbb: Embedded struct vb2_v4l2_buffer.
> >>> + * @list: List entry of the object for @struct mtk_cam_video_device:
> >>> + *        buf_list.
> >>> + * @daddr: The DMA address of this buffer.
> >>> + * @scp_addr: The SCP address of this buffer which
> >>> + *            is only supported for meta input node.
> >>> + * @node_id: The vidoe node id which this buffer belongs to.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_buffer {
> >>> +struct vb2_v4l2_buffer vbb;
> >>> +struct list_head list;
> >>> +/* Intenal part */
> >>> +dma_addr_t daddr;
> >>> +dma_addr_t scp_addr;
> >>> +unsigned int node_id;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
> >>> + *
> >>> + * @id: id of the node
> >>> + * @name: name of the node
> >>> + * @cap: supported V4L2 capabilities
> >>> + * @buf_type: supported V4L2 buffer type
> >>> + * @dma_port: the dma ports associated to the node
> >>> + * @link_flags: default media link flags
> >>> + * @smem_alloc: using the smem_dev as alloc device or not
> >>> + * @image: true for image node, false for meta node
> >>> + * @num_fmts: the number of supported node formats
> >>> + * @default_fmt_idx: default format of this node
> >>> + * @max_buf_count: maximum VB2 buffer count
> >>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
> >>> + * @fmts: supported format
> >>> + * @frmsizes: supported V4L2 frame size number
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev_node_desc {
> >>> +u8 id;
> >>> +const char *name;
> >>> +u32 cap;
> >>> +u32 buf_type;
> >>> +u32 dma_port;
> >>> +u32 link_flags;
> >>> +u8 smem_alloc:1;
> >>> +u8 image:1;
> >>> +u8 num_fmts;
> >>> +u8 default_fmt_idx;
> >>> +u8 max_buf_count;
> >>> +const struct v4l2_ioctl_ops *ioctl_ops;
> >>> +const struct v4l2_format *fmts;
> >>> +const struct v4l2_frmsizeenum *frmsizes;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_video_device - Mediatek video device structure
> >>> + *
> >>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
> >>> + * @enabled: Indicate the video device is enabled or not
> >>> + * @desc: The node description of video device
> >>> + * @vdev_fmt: The V4L2 format of video device
> >>> + * @vdev_pad: The media pad graph object of video device
> >>> + * @vbq: A videobuf queue of video device
> >>> + * @vdev: The video device instance
> >>> + * @vdev_lock: Serializes vb2 queue and video device operations
> >>> + * @buf_list: List for enqueue buffers
> >>> + * @buf_list_lock: Lock used to protect buffer list.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_video_device {
> >>> +unsigned int id;
> >>> +unsigned int enabled;
> >>> +struct mtk_cam_dev_node_desc desc;
> >>> +struct v4l2_format vdev_fmt;
> >>> +struct media_pad vdev_pad;
> >>> +struct vb2_queue vbq;
> >>> +struct video_device vdev;
> >>> +/* Serializes vb2 queue and video device operations */
> >>> +struct mutex vdev_lock;
> >>> +struct list_head buf_list;
> >>> +/* Lock used to protect buffer list */
> >>> +spinlock_t buf_list_lock;
> >>> +};
> >>> +
> >>> +/*
> >>> + * struct mtk_cam_dev - Mediatek camera device structure.
> >>> + *
> >>> + * @dev: Pointer to device.
> >>> + * @smem_pdev: Pointer to shared memory device.
> >>> + * @pipeline: Media pipeline information.
> >>> + * @media_dev: Media device instance.
> >>> + * @subdev: The V4L2 sub-device instance.
> >>> + * @v4l2_dev: The V4L2 device driver instance.
> >>> + * @notifier: The v4l2_device notifier data.
> >>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
> >>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
> >>> + * @seninf: Pointer to the seninf sub-device.
> >>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
> >>> + * @streaming: Indicate the overall streaming status is on or off.
> >>> + * @enabled_dmas: The enabled dma port information when streaming on.
> >>> + * @enabled_count: Number of enabled video nodes
> >>> + * @stream_count: Number of streaming video nodes
> >>> + * @running_job_count: Nunber of running jobs in the HW driver.
> >>> + * @pending_job_list: List to keep the media requests before en-queue into
> >>> + *                    HW driver.
> >>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
> >>> + * @running_job_list: List to keep the media requests after en-queue into
> >>> + *                    HW driver.
> >>> + * @running_job_lock: Protect the running_job_list data.
> >>> + * @op_lock: Serializes driver's VB2 callback operations.
> >>> + *
> >>> + */
> >>> +struct mtk_cam_dev {
> >>> +struct device *dev;
> >>> +struct device *smem_dev;
> >>> +struct media_pipeline pipeline;
> >>> +struct media_device media_dev;
> >>> +struct v4l2_subdev subdev;
> >>> +struct v4l2_device v4l2_dev;
> >>> +struct v4l2_async_notifier notifier;
> >>> +struct media_pad *subdev_pads;
> >>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
> >>> +struct v4l2_subdev *seninf;
> >>> +struct v4l2_subdev *sensor;
> >>> +unsigned int streaming;
> >>> +unsigned int enabled_dmas;
> >>> +unsigned int enabled_count;
> >>> +unsigned int stream_count;
> >>> +unsigned int running_job_count;
> >>> +struct list_head pending_job_list;
> >>> +/* Protect the pending_job_list data */
> >>> +spinlock_t pending_job_lock;
> >>> +struct list_head running_job_list;
> >>> +/* Protect the running_job_list data & running_job_count */
> >>> +spinlock_t running_job_lock;
> >>> +/* Serializes driver's VB2 callback operations */
> >>> +struct mutex op_lock;
> >>> +};
> >>> +
> >>> +int mtk_cam_dev_init(struct platform_device *pdev,
> >>> +     struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
> >>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
> >>> +   unsigned int frame_seq_no);
> >>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
> >>> +  unsigned int frame_seq_no);
> >>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
> >>> +unsigned int frame_seq_no);
> >>> +
> >>> +#endif /* __MTK_CAM_H__ */
> >>>
> >>
> >> _______________________________________________
> >> Linux-mediatek mailing list
> >> Linux-mediatek@lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> 
> Regards,
> Helen

Thanks for your comment.

Best regards,


Jungo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
       [not found]         ` <1fd3615eb18f48ada186bfe228fc907b@mtkmbs01n2.mediatek.inc>
@ 2020-05-04 12:40             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-05-04 12:40 UTC (permalink / raw)
  To: Helen Koike
  Cc: mchehab, shik, devicetree, Sean.Cheng, suleiman, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media, tfiga,
	hverkuil-cisco, laurent.pinchart, matthias.bgg


Hi Helen;

Sorry for late reply.
Please check my feedback & questions below.

On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> 
> Hi Jungo,
> 
> On 4/10/20 7:32 AM, Jungo Lin wrote:
> > Hi Helen:
> >
> > Thanks for your comment.
> >
> > On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >> Hi Jungo,
> >>
> >> I was taking a look at this patchset, please see my comments below.
> >>
> >> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>> Hello,
> >>>
> >>> This patch series adding the driver for Pass 1 (P1) unit in
> >>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> >>> camera features of CrOS.
> >>>
> >>> Pass 1 unit processes image signal from sensor devices and accepts the
> >>> tuning parameters to adjust the image quality. It performs optical
> >>> black correction, defect pixel correction, W/IR imbalance correction
> >>> and lens shading correction for RAW processing.
> >>>
> >>> The driver is implemented with V4L2 and media controller framework so
> >>> we have the following entities to describe the ISP pass 1 path.
> >>>
> >>> (The current metadata interface used in meta input and partial meta
> >>> nodes is only a temporary solution to kick off the driver development
> >>> and is not ready to be reviewed yet.)
> >>>
> >>> 1. meta input (output video device): connect to ISP P1 sub device.
> >>> It accepts the tuning buffer from user.
> >>>
> >>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> >>> main stream and packed out video devices. When processing an image,
> >>> Pass 1 hardware supports multiple output images with different sizes
> >>> and formats so it needs two capture video devices ("main stream" and
> >>> "packed out") to return the image data to the user.
> >>>
> >>> 3. main stream (capture video device): return the processed image data
> >>> which is used in capture scenario.
> >>>
> >>> 4. packed out (capture video device): return the processed image data
> >>> which is used in preview scenario.
> >>>
> >>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> >>>
> >>> 6. partial meta 1 (capture video device): return the AF statistics.
> >>>
> >>> 7. partial meta 2 (capture video device): return the local contrast
> >>>    enhanced statistics.
> >>>
> >>> 8. partial meta 3 (capture video device): return the local motion
> >>>    vector statistics.
> >>>
> >>> The overall patches of the series is:
> >>>
> >>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> >>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> >>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> >>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >>>   Moreover, implement standard V4L2 video driver that utilizes
> >>>   V4L2 and media framework APIs. Communicate with co-process via SCP
> >>>   communication to compose ISP registers in the firmware.
> >>>
> >>> Here is ISP P1 media topology:
> >>> It is included the main/sub sensor, sen-inf sub-devices and len device
> >>> which are implemented in below patch[1][2][3][4]:
> >>
> >> I would be nice if you could provide a branch with those applied.
> >>
> >
> > We apply those patches in the chromeos-4.19 to test.
> > https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> >
> >
> >>>
> >>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> >>> patch sets.
> >>>
> >>> /usr/bin/media-ctl -p -d /dev/media2
> >>>
> >>> Media controller API version 4.19.89
> >>>
> >>> Media device information
> >>> ------------------------
> >>> driver          mtk-cam-p1
> >>> model           mtk-cam-p1
> >>> serial
> >>> bus info        platform:1a000000.camisp
> >>> hw revision     0x0
> >>> driver version  4.19.89
> >>>
> >>> Device topology
> >>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
> >>
> >> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> >> Is this correct?
> >>
> >> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> >>
> >
> > Currently, we only support single-cam in this SoC with upstream driver.
> > It is plan in next Mediatek SoC to support multi-cam capabilities. So
> > we'd like to keep the naming to avoid confusion.
> 
> I suppose this new Mediatek SoC would use this same driver?
> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
> naming will be different then, right? (I guess it's ok).
> 

Sorry, my last comment should be corrected.
The new Mediatek SoC with new ISP HW version will support multi-cam
capabilities with new upstream driver. Due to the new enrich function,
it may not reuse current driver.

> >
> >>>             type V4L2 subdev subtype Unknown flags 0
> >>>             device node name /dev/v4l-subdev0
> >>> pad0: Sink
> >>> <- "mtk-cam-p1 meta input":0 []
> >>
> >> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> >>
> >
> > Ok, we will revise our naming in next patch.
> >
> >>> pad1: Source
> >>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> >>
> >> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> >>
> >
> > Yes, you are right. We will remove IMMUTABLE flag in next patch.
> >
> >>> pad2: Source
> >>> -> "mtk-cam-p1 packed out":0 []
> >>
> >> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> >>
> >
> > Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> > padding in the memory. We may revise the naming in next patch.
> >
> >>> pad3: Source
> >>> -> "mtk-cam-p1 partial meta 0":0 []
> >>> pad4: Source
> >>> -> "mtk-cam-p1 partial meta 1":0 []
> >>> pad5: Source
> >>> -> "mtk-cam-p1 partial meta 2":0 []
> >>> pad6: Source
> >>> -> "mtk-cam-p1 partial meta 3":0 []
> >>
> >> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> >>
> >> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> >> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> >>
> >> I also would prefer to remove blank spaces.
> >>
> >> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> >>
> >
> > No, these links are optional to setup for userspace.
> 
> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
> 
> I'm not sure if using the link state is the best option (please see my comment on 5/5).
> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
> interesting nodes at once.
> 
> 
> Regards,
> Helen
> 


According to your suggestion, do you have sample code about "userspace
could do a request to enable the stream in all the interesting nodes at
once"? If this supports, it is helpful for us to simply our current
implementation.


Thanks,


Jungo


> > For naming part, we will align with driver source codes.
> >
> >>> pad7: Source
> >>> pad8: Source
> >>> pad9: Source
> >>> pad10: Source
> >>
> >> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> >>
> >
> > These pads are just reserved purpose.
> > We will plan to remove them in next patch.
> >
> > Thanks,
> >
> > Jungo
> >
> >> Regards,
> >> Helen
> >>
> >>> pad11: Sink
> >>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video2
> >>> pad0: Source
> >>> -> "mtk-cam-p1":0 []
> >>>
> >>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video3
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video4
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":2 []
> >>>
> >>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video5
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":3 []
> >>>
> >>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video6
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":4 []
> >>>
> >>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video7
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":5 []
> >>>
> >>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video8
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":6 []
> >>>
> >>> - entity 56: 1a040000.seninf (12 pads, 3 links)
> >>>              type V4L2 subdev subtype Unknown flags 0
> >>>              device node name /dev/v4l-subdev1
> >>> pad0: Sink
> >>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> >>> <- "ov8856 2-0010":0 [ENABLED]
> >>> pad1: Sink
> >>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> >>> <- "ov02a10 4-003d":0 []
> >>> pad2: Sink
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad3: Sink
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad4: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> >>> pad5: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad6: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad7: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad8: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad9: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad10: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad11: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>
> >>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >>>              type V4L2 subdev subtype Sensor flags 0
> >>>              device node name /dev/v4l-subdev2
> >>> pad0: Source
> >>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> >>> -> "1a040000.seninf":0 [ENABLED]
> >>>
> >>> - entity 73: dw9768 2-000c (0 pad, 0 link)
> >>>              type V4L2 subdev subtype Lens flags 0
> >>>              device node name /dev/v4l-subdev3
> >>>
> >>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >>>              type V4L2 subdev subtype Sensor flags 0
> >>>              device node name /dev/v4l-subdev4
> >>> pad0: Source
> >>> [fmt:SRGGB10_1X10/1600x1200 field:none]
> >>> -> "1a040000.seninf":1 []
> >>>
> >>> ===========
> >>> = history =
> >>> ===========
> >>>
> >>> version 6:
> >>>  - Add port node description in the dt-binding document and device tree
> >>>    by Tomasz Figa.
> >>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >>>    version.
> >>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >>>  - Correct auto suspend timer value for suspend/resume issue.
> >>>  - Increase IPI guard timer to 1 second to avoid false alarm command
> >>>    timeout event.
> >>>  - Fix KE due to no sen-inf sub-device.
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation.
> >>>  - Add rst documents for Mediatek meta formats.
> >>>  - New meta buffer structure design & re-factoring.
> >>>
> >>> version 5:
> >>>  - Fixed Rob's comment on dt-binding format
> >>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>    and new timestamp type in driver
> >>>  - Fix buffer en-queue timing issue in v4
> >>>  - Remove default link_notify callback function in mtk_cam_media_ops
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>
> >>> version 4:
> >>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >>>    patch[4]
> >>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >>>  - Extend Mediatek proprietary image formats to support bayer order
> >>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>
> >>> version 3:
> >>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >>>    reserved memory region. (Rob Herring)
> >>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >>>  - Revise ISP Pass1 Kconfig
> >>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >>>  - Fix kernel warning messages when running v4l2_compliance test
> >>>  - Move AFO buffer enqueue & de-queue from request API to non-request
> >>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >>>    declaration (Hans Verkuil)
> >>>    Split GET_BIN_INFO control into two controls to get width & height
> >>>    in-dependently (Hans Verkuil)
> >>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >>>    Fix Drew's comments which are addressed in v2 patch
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>    Fix Drew's comments which are addressed in v2 patch
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>    Fix ISP de-initialize timing KE issue
> >>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> >>>
> >>> Todo:
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>
> >>> version 2:
> >>>  - Add 3A enhancement feature which includes:
> >>>    Separates 3A pipeline out of frame basis to improve
> >>>    AE/AWB (exposure and white balance) performance.
> >>>    Add 2 SCP sub-commands for 3A meta buffers.
> >>>  - Add new child device to manage P1 shared memory between P1 HW unit
> >>>    and co-processor.
> >>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >>>  - Revised document wording for dt-bindings documents & dts information.
> >>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >>>  - mtk_cam-dev.h / mtk_cam-dev.c
> >>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >>>    or add comments.
> >>>    Revised buffer size for LMVO & LCSO.
> >>>    Fix pixel format utility function.
> >>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >>>  - mtk_cam-v4l2-util.c
> >>>    Refactoring V4L2 async mechanism with seninf driver only
> >>>    Refactoring CIO (Connection IO) implementation with active sensor
> >>>    Revised stream on function for 3A enhancement feature
> >>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>    Add meta buffer index register definitions
> >>>    Add meta DMA configuration function.
> >>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >>>    Add isp_setup_scp_rproc function to get RPC handle
> >>>    Add mtk_cam_reserved_memory_init for shared memory management
> >>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>    Add new meta strictures for 3A enhancement feature
> >>>    Add new IPI command utility function for 3A enhancement feature
> >>>    Enhance isp_composer_dma_sg_init function flow
> >>>    Shorten overall IPI command structure size
> >>>    Remove scp_state state checking
> >>>    Improve code readability
> >>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >>>    memory region.
> >>>
> >>> TODOs:
> >>>  - 3A enhancement feature bug fixing
> >>>
> >>> version 1:
> >>>  - Revised driver sources based on Tomasz's comments including
> >>>    part1/2/3/4 in RFC V0 patch.
> >>>  - Remove DMA cache mechanism.
> >>>    Support two new video devices (LCSO/LMVO) for advance camera
> >>>    features.
> >>>  - Fixed v4l2-compliance test failure items.
> >>>  - Add private controls for Mediatek camera middle-ware.
> >>>  - Replace VPU driver's APIs with new SCP driver interface for
> >>>    co-processor communication.
> >>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >>>    commands RX handling.
> >>>  - Fix internal bugs.
> >>>
> >>> TODOs:
> >>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >>>    for shared memory management.
> >>>  - Revised file names.
> >>>  - Support non frame-sync AFO/AAO DMA buffers
> >>>
> >>> version 0:
> >>> - Initial submission
> >>>
> >>> ==================
> >>>  Dependent patch set
> >>> ==================
> >>>
> >>> Camera ISP P1 driver depends on seninf driver, SCP driver.
> >>> The patches are listed as following:
> >>>
> >>> [1]. media: support Mediatek sensor interface driver
> >>> https://patchwork.kernel.org/cover/11145845/
> >>>
> >>> [2]. media: ov8856: Add YAML binding and sensor mode support
> >>> https://patchwork.kernel.org/cover/11220785/
> >>>
> >>> [3]. media: i2c: Add support for OV02A10 sensor
> >>> https://patchwork.kernel.org/cover/11284779/
> >>>
> >>> [4]. media: i2c: add support for DW9768 VCM driver
> >>> https://patchwork.kernel.org/cover/11132299/
> >>>
> >>> [5]. Add support for mt8183 SCP
> >>> https://patchwork.kernel.org/cover/11239065/
> >>>
> >>> [6]. MT8183 IOMMU SUPPORT
> >>> https://patchwork.kernel.org/cover/11112765/
> >>>
> >>> ==================
> >>>  Compliance test
> >>> ==================
> >>>
> >>> The v4l2-compliance is built with the below lastest patch.
> >>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> >>>
> >>> Note 1.
> >>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> >>>
> >>> Note 2.
> >>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> >>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> >>>
> >>> Note 3.
> >>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> >>>
> >>> /usr/bin/v4l2-compliance -m /dev/media2
> >>>
> >>> v4l2-compliance SHA: not available, 32 bits
> >>>
> >>> Compliance test for mtk-cam-p1 device /dev/media1:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.67
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.67
> >>>
> >>> Required ioctls:
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/media1 open: OK
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Media Controller ioctls:
> >>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> >>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>> test MEDIA_IOC_SETUP_LINK: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video25:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.67
> >>> Capabilities     : 0x8c200000
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x0c200000
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.67
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.67
> >>> Interface Info:
> >>> ID               : 0x03000010
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000000e (14)
> >>> Name             : mtk-cam-p1 meta input
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100000f   : 0: Source
> >>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video25 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composiv4l2-compliance SHA: not available, 32 bits
> >>>
> >>> Compliance test for mtk-cam-p1 device /dev/media2:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>>
> >>> Required ioctls:
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/media2 open: OK
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Media Controller ioctls:
> >>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> >>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>> test MEDIA_IOC_SETUP_LINK: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video2:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x8c200000
> >>> Metadata Output
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x0c200000
> >>> Metadata Output
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000010
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000000e (14)
> >>> Name             : mtk-cam-p1 meta input
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100000f   : 0: Source
> >>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video2 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video3:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000016
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000014 (20)
> >>> Name             : mtk-cam-p1 main stream
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000015   : 0: Sink
> >>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video3 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video4:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x0300001c
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000001a (26)
> >>> Name             : mtk-cam-p1 packed out
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100001b   : 0: Sink
> >>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video4 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video5:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000022
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000020 (32)
> >>> Name             : mtk-cam-p1 partial meta 0
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000021   : 0: Sink
> >>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video5 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video6:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000028
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000026 (38)
> >>> Name             : mtk-cam-p1 partial meta 1
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000027   : 0: Sink
> >>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video6 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video7:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x0300002e
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000002c (44)
> >>> Name             : mtk-cam-p1 partial meta 2
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100002d   : 0: Sink
> >>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video7 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video8:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000034
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000032 (50)
> >>> Name             : mtk-cam-p1 partial meta 3
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000033   : 0: Sink
> >>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video8 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000050
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000001 (1)
> >>> Name             : mtk-cam-p1
> >>> Function         : Video Pixel Formatter
> >>> Pad 0x01000002   : 0: Sink
> >>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> >>> Pad 0x01000003   : 1: Source
> >>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> >>> Pad 0x01000004   : 2: Source
> >>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> >>> Pad 0x01000005   : 3: Source
> >>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> >>> Pad 0x01000006   : 4: Source
> >>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> >>> Pad 0x01000007   : 5: Source
> >>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> >>> Pad 0x01000008   : 6: Source
> >>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> >>> Pad 0x01000009   : 7: Source
> >>> Pad 0x0100000a   : 8: Source
> >>> Pad 0x0100000b   : 9: Source
> >>> Pad 0x0100000c   : 10: Source
> >>> Pad 0x0100000d   : 11: Sink
> >>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev0 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 1):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 2):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 3):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 4):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 5):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 6):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 7):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 8):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 9):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 10):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 11):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000052
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000038 (56)
> >>> Name             : 1a040000.seninf
> >>> Function         : Video Interface Bridge
> >>> Pad 0x01000039   : 0: Sink
> >>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> >>> Pad 0x0100003a   : 1: Sink
> >>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> >>> Pad 0x0100003b   : 2: Sink
> >>> Pad 0x0100003c   : 3: Sink
> >>> Pad 0x0100003d   : 4: Source
> >>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>> Pad 0x0100003e   : 5: Source
> >>> Pad 0x0100003f   : 6: Source
> >>> Pad 0x01000040   : 7: Source
> >>> Pad 0x01000041   : 8: Source
> >>> Pad 0x01000042   : 9: Source
> >>> Pad 0x01000043   : 10: Source
> >>> Pad 0x01000044   : 11: Source
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev1 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 1):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 2):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 3):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 4):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 5):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 6):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 7):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 8):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 9):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 10):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 11):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 2 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000054
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000045 (69)
> >>> Name             : ov8856 2-0010
> >>> Function         : Camera Sensor
> >>> Pad 0x01000046   : 0: Source
> >>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev2 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 0):
> >>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> >>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 11 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000056
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000049 (73)
> >>> Name             : dw9768 2-000c
> >>> Function         : Lens Controller
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev3 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 2 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000058
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x0000004a (74)
> >>> Name             : ov02a10 4-003d
> >>> Function         : Camera Sensor
> >>> Pad 0x0100004b   : 0: Source
> >>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev4 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> fail: v4l2-test-controls.cpp(362): returned control value out of range
> >>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> >>> test VIDIOC_G/S_CTRL: FAIL
> >>> fail: v4l2-test-controls.cpp(549): returned control value out of range
> >>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 10 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> >>>
> >>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> >>>
> >>>
> >>> Jungo Lin (5):
> >>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >>>   media: videodev2.h: Add new boottime timestamp type
> >>>   media: platform: Add Mediatek ISP P1 image & meta formats
> >>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> >>>
> >>>  .../bindings/media/mediatek,camisp.txt        |   83 +
> >>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >>>  drivers/media/platform/Kconfig                |    1 +
> >>>  drivers/media/platform/Makefile               |    1 +
> >>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >>>  include/uapi/linux/videodev2.h                |   41 +
> >>>  24 files changed, 4226 insertions(+), 1 deletion(-)
> >>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>>
> >>
> >> _______________________________________________
> >> Linux-mediatek mailing list
> >> Linux-mediatek@lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-mediatek



^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-05-04 12:40             ` Jungo Lin
  0 siblings, 0 replies; 388+ messages in thread
From: Jungo Lin @ 2020-05-04 12:40 UTC (permalink / raw)
  To: Helen Koike
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	ddavenport, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, hverkuil-cisco


Hi Helen;

Sorry for late reply.
Please check my feedback & questions below.

On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> 
> Hi Jungo,
> 
> On 4/10/20 7:32 AM, Jungo Lin wrote:
> > Hi Helen:
> >
> > Thanks for your comment.
> >
> > On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >> Hi Jungo,
> >>
> >> I was taking a look at this patchset, please see my comments below.
> >>
> >> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>> Hello,
> >>>
> >>> This patch series adding the driver for Pass 1 (P1) unit in
> >>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> >>> camera features of CrOS.
> >>>
> >>> Pass 1 unit processes image signal from sensor devices and accepts the
> >>> tuning parameters to adjust the image quality. It performs optical
> >>> black correction, defect pixel correction, W/IR imbalance correction
> >>> and lens shading correction for RAW processing.
> >>>
> >>> The driver is implemented with V4L2 and media controller framework so
> >>> we have the following entities to describe the ISP pass 1 path.
> >>>
> >>> (The current metadata interface used in meta input and partial meta
> >>> nodes is only a temporary solution to kick off the driver development
> >>> and is not ready to be reviewed yet.)
> >>>
> >>> 1. meta input (output video device): connect to ISP P1 sub device.
> >>> It accepts the tuning buffer from user.
> >>>
> >>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> >>> main stream and packed out video devices. When processing an image,
> >>> Pass 1 hardware supports multiple output images with different sizes
> >>> and formats so it needs two capture video devices ("main stream" and
> >>> "packed out") to return the image data to the user.
> >>>
> >>> 3. main stream (capture video device): return the processed image data
> >>> which is used in capture scenario.
> >>>
> >>> 4. packed out (capture video device): return the processed image data
> >>> which is used in preview scenario.
> >>>
> >>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> >>>
> >>> 6. partial meta 1 (capture video device): return the AF statistics.
> >>>
> >>> 7. partial meta 2 (capture video device): return the local contrast
> >>>    enhanced statistics.
> >>>
> >>> 8. partial meta 3 (capture video device): return the local motion
> >>>    vector statistics.
> >>>
> >>> The overall patches of the series is:
> >>>
> >>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> >>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> >>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> >>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >>>   Moreover, implement standard V4L2 video driver that utilizes
> >>>   V4L2 and media framework APIs. Communicate with co-process via SCP
> >>>   communication to compose ISP registers in the firmware.
> >>>
> >>> Here is ISP P1 media topology:
> >>> It is included the main/sub sensor, sen-inf sub-devices and len device
> >>> which are implemented in below patch[1][2][3][4]:
> >>
> >> I would be nice if you could provide a branch with those applied.
> >>
> >
> > We apply those patches in the chromeos-4.19 to test.
> > https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> >
> >
> >>>
> >>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> >>> patch sets.
> >>>
> >>> /usr/bin/media-ctl -p -d /dev/media2
> >>>
> >>> Media controller API version 4.19.89
> >>>
> >>> Media device information
> >>> ------------------------
> >>> driver          mtk-cam-p1
> >>> model           mtk-cam-p1
> >>> serial
> >>> bus info        platform:1a000000.camisp
> >>> hw revision     0x0
> >>> driver version  4.19.89
> >>>
> >>> Device topology
> >>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
> >>
> >> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> >> Is this correct?
> >>
> >> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> >>
> >
> > Currently, we only support single-cam in this SoC with upstream driver.
> > It is plan in next Mediatek SoC to support multi-cam capabilities. So
> > we'd like to keep the naming to avoid confusion.
> 
> I suppose this new Mediatek SoC would use this same driver?
> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
> naming will be different then, right? (I guess it's ok).
> 

Sorry, my last comment should be corrected.
The new Mediatek SoC with new ISP HW version will support multi-cam
capabilities with new upstream driver. Due to the new enrich function,
it may not reuse current driver.

> >
> >>>             type V4L2 subdev subtype Unknown flags 0
> >>>             device node name /dev/v4l-subdev0
> >>> pad0: Sink
> >>> <- "mtk-cam-p1 meta input":0 []
> >>
> >> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> >>
> >
> > Ok, we will revise our naming in next patch.
> >
> >>> pad1: Source
> >>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> >>
> >> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> >>
> >
> > Yes, you are right. We will remove IMMUTABLE flag in next patch.
> >
> >>> pad2: Source
> >>> -> "mtk-cam-p1 packed out":0 []
> >>
> >> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> >>
> >
> > Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> > padding in the memory. We may revise the naming in next patch.
> >
> >>> pad3: Source
> >>> -> "mtk-cam-p1 partial meta 0":0 []
> >>> pad4: Source
> >>> -> "mtk-cam-p1 partial meta 1":0 []
> >>> pad5: Source
> >>> -> "mtk-cam-p1 partial meta 2":0 []
> >>> pad6: Source
> >>> -> "mtk-cam-p1 partial meta 3":0 []
> >>
> >> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> >>
> >> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> >> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> >>
> >> I also would prefer to remove blank spaces.
> >>
> >> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> >>
> >
> > No, these links are optional to setup for userspace.
> 
> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
> 
> I'm not sure if using the link state is the best option (please see my comment on 5/5).
> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
> interesting nodes at once.
> 
> 
> Regards,
> Helen
> 


According to your suggestion, do you have sample code about "userspace
could do a request to enable the stream in all the interesting nodes at
once"? If this supports, it is helpful for us to simply our current
implementation.


Thanks,


Jungo


> > For naming part, we will align with driver source codes.
> >
> >>> pad7: Source
> >>> pad8: Source
> >>> pad9: Source
> >>> pad10: Source
> >>
> >> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> >>
> >
> > These pads are just reserved purpose.
> > We will plan to remove them in next patch.
> >
> > Thanks,
> >
> > Jungo
> >
> >> Regards,
> >> Helen
> >>
> >>> pad11: Sink
> >>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video2
> >>> pad0: Source
> >>> -> "mtk-cam-p1":0 []
> >>>
> >>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video3
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video4
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":2 []
> >>>
> >>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video5
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":3 []
> >>>
> >>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video6
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":4 []
> >>>
> >>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video7
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":5 []
> >>>
> >>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >>>              type Node subtype V4L flags 0
> >>>              device node name /dev/video8
> >>> pad0: Sink
> >>> <- "mtk-cam-p1":6 []
> >>>
> >>> - entity 56: 1a040000.seninf (12 pads, 3 links)
> >>>              type V4L2 subdev subtype Unknown flags 0
> >>>              device node name /dev/v4l-subdev1
> >>> pad0: Sink
> >>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> >>> <- "ov8856 2-0010":0 [ENABLED]
> >>> pad1: Sink
> >>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> >>> <- "ov02a10 4-003d":0 []
> >>> pad2: Sink
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad3: Sink
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad4: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> >>> pad5: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad6: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad7: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad8: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad9: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad10: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>> pad11: Source
> >>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>
> >>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >>>              type V4L2 subdev subtype Sensor flags 0
> >>>              device node name /dev/v4l-subdev2
> >>> pad0: Source
> >>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> >>> -> "1a040000.seninf":0 [ENABLED]
> >>>
> >>> - entity 73: dw9768 2-000c (0 pad, 0 link)
> >>>              type V4L2 subdev subtype Lens flags 0
> >>>              device node name /dev/v4l-subdev3
> >>>
> >>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >>>              type V4L2 subdev subtype Sensor flags 0
> >>>              device node name /dev/v4l-subdev4
> >>> pad0: Source
> >>> [fmt:SRGGB10_1X10/1600x1200 field:none]
> >>> -> "1a040000.seninf":1 []
> >>>
> >>> ===========
> >>> = history =
> >>> ===========
> >>>
> >>> version 6:
> >>>  - Add port node description in the dt-binding document and device tree
> >>>    by Tomasz Figa.
> >>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >>>    version.
> >>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >>>  - Correct auto suspend timer value for suspend/resume issue.
> >>>  - Increase IPI guard timer to 1 second to avoid false alarm command
> >>>    timeout event.
> >>>  - Fix KE due to no sen-inf sub-device.
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation.
> >>>  - Add rst documents for Mediatek meta formats.
> >>>  - New meta buffer structure design & re-factoring.
> >>>
> >>> version 5:
> >>>  - Fixed Rob's comment on dt-binding format
> >>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>    and new timestamp type in driver
> >>>  - Fix buffer en-queue timing issue in v4
> >>>  - Remove default link_notify callback function in mtk_cam_media_ops
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>
> >>> version 4:
> >>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >>>    patch[4]
> >>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >>>  - Extend Mediatek proprietary image formats to support bayer order
> >>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> >>>
> >>> Todo:
> >>>  - vb2_ops's buf_request_complete callback function implementation
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>
> >>> version 3:
> >>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >>>    reserved memory region. (Rob Herring)
> >>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >>>  - Revise ISP Pass1 Kconfig
> >>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >>>  - Fix kernel warning messages when running v4l2_compliance test
> >>>  - Move AFO buffer enqueue & de-queue from request API to non-request
> >>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >>>    declaration (Hans Verkuil)
> >>>    Split GET_BIN_INFO control into two controls to get width & height
> >>>    in-dependently (Hans Verkuil)
> >>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >>>    Fix Drew's comments which are addressed in v2 patch
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>    Fix Drew's comments which are addressed in v2 patch
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>    Fix ISP de-initialize timing KE issue
> >>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> >>>
> >>> Todo:
> >>>  - Add rst documents for Mediatek meta formats
> >>>  - New meta buffer structure design & re-factoring
> >>>
> >>> version 2:
> >>>  - Add 3A enhancement feature which includes:
> >>>    Separates 3A pipeline out of frame basis to improve
> >>>    AE/AWB (exposure and white balance) performance.
> >>>    Add 2 SCP sub-commands for 3A meta buffers.
> >>>  - Add new child device to manage P1 shared memory between P1 HW unit
> >>>    and co-processor.
> >>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >>>  - Revised document wording for dt-bindings documents & dts information.
> >>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >>>  - mtk_cam-dev.h / mtk_cam-dev.c
> >>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >>>    or add comments.
> >>>    Revised buffer size for LMVO & LCSO.
> >>>    Fix pixel format utility function.
> >>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >>>  - mtk_cam-v4l2-util.c
> >>>    Refactoring V4L2 async mechanism with seninf driver only
> >>>    Refactoring CIO (Connection IO) implementation with active sensor
> >>>    Revised stream on function for 3A enhancement feature
> >>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>    Add meta buffer index register definitions
> >>>    Add meta DMA configuration function.
> >>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >>>    Add isp_setup_scp_rproc function to get RPC handle
> >>>    Add mtk_cam_reserved_memory_init for shared memory management
> >>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>    Add new meta strictures for 3A enhancement feature
> >>>    Add new IPI command utility function for 3A enhancement feature
> >>>    Enhance isp_composer_dma_sg_init function flow
> >>>    Shorten overall IPI command structure size
> >>>    Remove scp_state state checking
> >>>    Improve code readability
> >>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >>>    memory region.
> >>>
> >>> TODOs:
> >>>  - 3A enhancement feature bug fixing
> >>>
> >>> version 1:
> >>>  - Revised driver sources based on Tomasz's comments including
> >>>    part1/2/3/4 in RFC V0 patch.
> >>>  - Remove DMA cache mechanism.
> >>>    Support two new video devices (LCSO/LMVO) for advance camera
> >>>    features.
> >>>  - Fixed v4l2-compliance test failure items.
> >>>  - Add private controls for Mediatek camera middle-ware.
> >>>  - Replace VPU driver's APIs with new SCP driver interface for
> >>>    co-processor communication.
> >>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >>>    commands RX handling.
> >>>  - Fix internal bugs.
> >>>
> >>> TODOs:
> >>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >>>    for shared memory management.
> >>>  - Revised file names.
> >>>  - Support non frame-sync AFO/AAO DMA buffers
> >>>
> >>> version 0:
> >>> - Initial submission
> >>>
> >>> ==================
> >>>  Dependent patch set
> >>> ==================
> >>>
> >>> Camera ISP P1 driver depends on seninf driver, SCP driver.
> >>> The patches are listed as following:
> >>>
> >>> [1]. media: support Mediatek sensor interface driver
> >>> https://patchwork.kernel.org/cover/11145845/
> >>>
> >>> [2]. media: ov8856: Add YAML binding and sensor mode support
> >>> https://patchwork.kernel.org/cover/11220785/
> >>>
> >>> [3]. media: i2c: Add support for OV02A10 sensor
> >>> https://patchwork.kernel.org/cover/11284779/
> >>>
> >>> [4]. media: i2c: add support for DW9768 VCM driver
> >>> https://patchwork.kernel.org/cover/11132299/
> >>>
> >>> [5]. Add support for mt8183 SCP
> >>> https://patchwork.kernel.org/cover/11239065/
> >>>
> >>> [6]. MT8183 IOMMU SUPPORT
> >>> https://patchwork.kernel.org/cover/11112765/
> >>>
> >>> ==================
> >>>  Compliance test
> >>> ==================
> >>>
> >>> The v4l2-compliance is built with the below lastest patch.
> >>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> >>>
> >>> Note 1.
> >>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> >>>
> >>> Note 2.
> >>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> >>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> >>>
> >>> Note 3.
> >>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> >>>
> >>> /usr/bin/v4l2-compliance -m /dev/media2
> >>>
> >>> v4l2-compliance SHA: not available, 32 bits
> >>>
> >>> Compliance test for mtk-cam-p1 device /dev/media1:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.67
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.67
> >>>
> >>> Required ioctls:
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/media1 open: OK
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Media Controller ioctls:
> >>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> >>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>> test MEDIA_IOC_SETUP_LINK: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video25:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.67
> >>> Capabilities     : 0x8c200000
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x0c200000
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.67
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.67
> >>> Interface Info:
> >>> ID               : 0x03000010
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000000e (14)
> >>> Name             : mtk-cam-p1 meta input
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100000f   : 0: Source
> >>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video25 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composiv4l2-compliance SHA: not available, 32 bits
> >>>
> >>> Compliance test for mtk-cam-p1 device /dev/media2:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>>
> >>> Required ioctls:
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/media2 open: OK
> >>> test MEDIA_IOC_DEVICE_INFO: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Media Controller ioctls:
> >>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> >>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>> test MEDIA_IOC_SETUP_LINK: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video2:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x8c200000
> >>> Metadata Output
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x0c200000
> >>> Metadata Output
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000010
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000000e (14)
> >>> Name             : mtk-cam-p1 meta input
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100000f   : 0: Source
> >>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video2 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video3:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000016
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000014 (20)
> >>> Name             : mtk-cam-p1 main stream
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000015   : 0: Sink
> >>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video3 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video4:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04201000
> >>> Video Capture Multiplanar
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x0300001c
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000001a (26)
> >>> Name             : mtk-cam-p1 packed out
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100001b   : 0: Sink
> >>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video4 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video5:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000022
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000020 (32)
> >>> Name             : mtk-cam-p1 partial meta 0
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000021   : 0: Sink
> >>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video5 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video6:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000028
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000026 (38)
> >>> Name             : mtk-cam-p1 partial meta 1
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000027   : 0: Sink
> >>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video6 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video7:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x0300002e
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x0000002c (44)
> >>> Name             : mtk-cam-p1 partial meta 2
> >>> Function         : V4L2 I/O
> >>> Pad 0x0100002d   : 0: Sink
> >>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video7 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/video8:
> >>>
> >>> Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Card type        : mtk-cam-p1
> >>> Bus info         : platform:1a000000.camisp
> >>> Driver version   : 4.19.89
> >>> Capabilities     : 0x84a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Device Capabilities
> >>> Device Caps      : 0x04a00000
> >>> Metadata Capture
> >>> Streaming
> >>> Extended Pix Format
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000034
> >>> Type             : V4L Video
> >>> Entity Info:
> >>> ID               : 0x00000032 (50)
> >>> Name             : mtk-cam-p1 partial meta 3
> >>> Function         : V4L2 I/O
> >>> Pad 0x01000033   : 0: Sink
> >>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>> test VIDIOC_QUERYCAP: OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/video8 open: OK
> >>> test VIDIOC_QUERYCAP: OK
> >>> test VIDIOC_G/S_PRIORITY: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK
> >>> test VIDIOC_TRY_FMT: OK
> >>> test VIDIOC_S_FMT: OK
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK
> >>>
> >>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000050
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000001 (1)
> >>> Name             : mtk-cam-p1
> >>> Function         : Video Pixel Formatter
> >>> Pad 0x01000002   : 0: Sink
> >>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> >>> Pad 0x01000003   : 1: Source
> >>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> >>> Pad 0x01000004   : 2: Source
> >>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> >>> Pad 0x01000005   : 3: Source
> >>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> >>> Pad 0x01000006   : 4: Source
> >>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> >>> Pad 0x01000007   : 5: Source
> >>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> >>> Pad 0x01000008   : 6: Source
> >>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> >>> Pad 0x01000009   : 7: Source
> >>> Pad 0x0100000a   : 8: Source
> >>> Pad 0x0100000b   : 9: Source
> >>> Pad 0x0100000c   : 10: Source
> >>> Pad 0x0100000d   : 11: Sink
> >>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev0 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 1):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 2):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 3):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 4):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 5):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 6):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 7):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 8):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 9):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 10):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 11):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 0 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000052
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000038 (56)
> >>> Name             : 1a040000.seninf
> >>> Function         : Video Interface Bridge
> >>> Pad 0x01000039   : 0: Sink
> >>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> >>> Pad 0x0100003a   : 1: Sink
> >>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> >>> Pad 0x0100003b   : 2: Sink
> >>> Pad 0x0100003c   : 3: Sink
> >>> Pad 0x0100003d   : 4: Source
> >>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>> Pad 0x0100003e   : 5: Source
> >>> Pad 0x0100003f   : 6: Source
> >>> Pad 0x01000040   : 7: Source
> >>> Pad 0x01000041   : 8: Source
> >>> Pad 0x01000042   : 9: Source
> >>> Pad 0x01000043   : 10: Source
> >>> Pad 0x01000044   : 11: Source
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev1 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 1):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 2):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Sink Pad 3):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 4):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 5):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 6):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 7):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 8):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 9):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 10):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 11):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 2 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000054
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000045 (69)
> >>> Name             : ov8856 2-0010
> >>> Function         : Camera Sensor
> >>> Pad 0x01000046   : 0: Source
> >>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev2 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 0):
> >>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> >>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 11 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000056
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x00000049 (73)
> >>> Name             : dw9768 2-000c
> >>> Function         : Lens Controller
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev3 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> test VIDIOC_G/S_CTRL: OK
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 2 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> >>> --------------------------------------------------------------------------------
> >>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> >>>
> >>> Media Driver Info:
> >>> Driver name      : mtk-cam-p1
> >>> Model            : mtk-cam-p1
> >>> Serial           :
> >>> Bus info         : platform:1a000000.camisp
> >>> Media version    : 4.19.89
> >>> Hardware revision: 0x00000000 (0)
> >>> Driver version   : 4.19.89
> >>> Interface Info:
> >>> ID               : 0x03000058
> >>> Type             : V4L Sub-Device
> >>> Entity Info:
> >>> ID               : 0x0000004a (74)
> >>> Name             : ov02a10 4-003d
> >>> Function         : Camera Sensor
> >>> Pad 0x0100004b   : 0: Source
> >>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> >>>
> >>> Required ioctls:
> >>> test MC information (see 'Media Driver Info' above): OK
> >>>
> >>> Allow for multiple opens:
> >>> test second /dev/v4l-subdev4 open: OK
> >>> test for unlimited opens: OK
> >>>
> >>> Debug ioctls:
> >>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>
> >>> Input ioctls:
> >>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>
> >>> Output ioctls:
> >>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>
> >>> Input/Output configuration ioctls:
> >>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>
> >>> Sub-Device ioctls (Source Pad 0):
> >>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>
> >>> Control ioctls:
> >>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>> test VIDIOC_QUERYCTRL: OK
> >>> fail: v4l2-test-controls.cpp(362): returned control value out of range
> >>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> >>> test VIDIOC_G/S_CTRL: FAIL
> >>> fail: v4l2-test-controls.cpp(549): returned control value out of range
> >>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> >>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>> Standard Controls: 10 Private Controls: 0
> >>>
> >>> Format ioctls:
> >>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>> test VIDIOC_G_FMT: OK (Not Supported)
> >>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>> test VIDIOC_S_FMT: OK (Not Supported)
> >>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>> test Cropping: OK (Not Supported)
> >>> test Composing: OK (Not Supported)
> >>> test Scaling: OK (Not Supported)
> >>>
> >>> Codec ioctls:
> >>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>
> >>> Buffer ioctls:
> >>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>> test Requests: OK (Not Supported)
> >>>
> >>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> >>>
> >>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> >>>
> >>>
> >>> Jungo Lin (5):
> >>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >>>   media: videodev2.h: Add new boottime timestamp type
> >>>   media: platform: Add Mediatek ISP P1 image & meta formats
> >>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> >>>
> >>>  .../bindings/media/mediatek,camisp.txt        |   83 +
> >>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >>>  drivers/media/platform/Kconfig                |    1 +
> >>>  drivers/media/platform/Makefile               |    1 +
> >>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >>>  include/uapi/linux/videodev2.h                |   41 +
> >>>  24 files changed, 4226 insertions(+), 1 deletion(-)
> >>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>>
> >>
> >> _______________________________________________
> >> Linux-mediatek mailing list
> >> Linux-mediatek@lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-mediatek


_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
  2020-05-04 12:40             ` Jungo Lin
  (?)
@ 2020-05-05 15:30               ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: mchehab, shik, devicetree, Sean.Cheng, suleiman, Pi-Hsun Shih,
	srv_heupstream, robh, ryan.yu, Jerry-ch.Chen, frankie.chiu,
	sj.huang, yuzhao, linux-mediatek, zwisler, ddavenport,
	frederic.chen, linux-arm-kernel, linux-media, tfiga,
	hverkuil-cisco, laurent.pinchart, matthias.bgg

Hi Jungo,

On 5/4/20 9:40 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>>
>> Hi Jungo,
>>
>> On 4/10/20 7:32 AM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comment.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hi Jungo,
>>>>
>>>> I was taking a look at this patchset, please see my comments below.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> Hello,
>>>>>
>>>>> This patch series adding the driver for Pass 1 (P1) unit in
>>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>>>> camera features of CrOS.
>>>>>
>>>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>>>> tuning parameters to adjust the image quality. It performs optical
>>>>> black correction, defect pixel correction, W/IR imbalance correction
>>>>> and lens shading correction for RAW processing.
>>>>>
>>>>> The driver is implemented with V4L2 and media controller framework so
>>>>> we have the following entities to describe the ISP pass 1 path.
>>>>>
>>>>> (The current metadata interface used in meta input and partial meta
>>>>> nodes is only a temporary solution to kick off the driver development
>>>>> and is not ready to be reviewed yet.)
>>>>>
>>>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>>>> It accepts the tuning buffer from user.
>>>>>
>>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>>>> main stream and packed out video devices. When processing an image,
>>>>> Pass 1 hardware supports multiple output images with different sizes
>>>>> and formats so it needs two capture video devices ("main stream" and
>>>>> "packed out") to return the image data to the user.
>>>>>
>>>>> 3. main stream (capture video device): return the processed image data
>>>>> which is used in capture scenario.
>>>>>
>>>>> 4. packed out (capture video device): return the processed image data
>>>>> which is used in preview scenario.
>>>>>
>>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>>>
>>>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>>>
>>>>> 7. partial meta 2 (capture video device): return the local contrast
>>>>>    enhanced statistics.
>>>>>
>>>>> 8. partial meta 3 (capture video device): return the local motion
>>>>>    vector statistics.
>>>>>
>>>>> The overall patches of the series is:
>>>>>
>>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>>>   communication to compose ISP registers in the firmware.
>>>>>
>>>>> Here is ISP P1 media topology:
>>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>>>> which are implemented in below patch[1][2][3][4]:
>>>>
>>>> I would be nice if you could provide a branch with those applied.
>>>>
>>>
>>> We apply those patches in the chromeos-4.19 to test.
>>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
>>>
>>>
>>>>>
>>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>>>> patch sets.
>>>>>
>>>>> /usr/bin/media-ctl -p -d /dev/media2
>>>>>
>>>>> Media controller API version 4.19.89
>>>>>
>>>>> Media device information
>>>>> ------------------------
>>>>> driver          mtk-cam-p1
>>>>> model           mtk-cam-p1
>>>>> serial
>>>>> bus info        platform:1a000000.camisp
>>>>> hw revision     0x0
>>>>> driver version  4.19.89
>>>>>
>>>>> Device topology
>>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>>>
>>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>>>> Is this correct?
>>>>
>>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>>>
>>>
>>> Currently, we only support single-cam in this SoC with upstream driver.
>>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
>>> we'd like to keep the naming to avoid confusion.
>>
>> I suppose this new Mediatek SoC would use this same driver?
>> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
>> naming will be different then, right? (I guess it's ok).
>>
> 
> Sorry, my last comment should be corrected.
> The new Mediatek SoC with new ISP HW version will support multi-cam
> capabilities with new upstream driver. Due to the new enrich function,
> it may not reuse current driver.

right, thanks for the clarification.

> 
>>>
>>>>>             type V4L2 subdev subtype Unknown flags 0
>>>>>             device node name /dev/v4l-subdev0
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1 meta input":0 []
>>>>
>>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>>>
>>>
>>> Ok, we will revise our naming in next patch.
>>>
>>>>> pad1: Source
>>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>>>
>>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>>>
>>>
>>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
>>>
>>>>> pad2: Source
>>>>> -> "mtk-cam-p1 packed out":0 []
>>>>
>>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>>>
>>>
>>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
>>> padding in the memory. We may revise the naming in next patch.
>>>
>>>>> pad3: Source
>>>>> -> "mtk-cam-p1 partial meta 0":0 []
>>>>> pad4: Source
>>>>> -> "mtk-cam-p1 partial meta 1":0 []
>>>>> pad5: Source
>>>>> -> "mtk-cam-p1 partial meta 2":0 []
>>>>> pad6: Source
>>>>> -> "mtk-cam-p1 partial meta 3":0 []
>>>>
>>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>>>
>>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>>>
>>>> I also would prefer to remove blank spaces.
>>>>
>>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>>>
>>>
>>> No, these links are optional to setup for userspace.
>>
>> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
>> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
>>
>> I'm not sure if using the link state is the best option (please see my comment on 5/5).
>> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
>> interesting nodes at once.
>>
>>
>> Regards,
>> Helen
>>
> 
> 
> According to your suggestion, do you have sample code about "userspace
> could do a request to enable the stream in all the interesting nodes at
> once"? If this supports, it is helpful for us to simply our current
> implementation.

I was checking the request api docs [1] in more details and it seems this isn't possible, since
"A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
on MEDIA_IOC_REQUEST_ALLOC doc[2].

[1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
[2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc

What bothers me with the current implementation is that users could correctly configure the topology, but when
calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
for userspace about what is wrong.

So I was trying to think about how this could be done differently.
I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

Regards,
Helen


> 
> 
> Thanks,
> 
> 
> Jungo
> 
> 
>>> For naming part, we will align with driver source codes.
>>>
>>>>> pad7: Source
>>>>> pad8: Source
>>>>> pad9: Source
>>>>> pad10: Source
>>>>
>>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>>>
>>>
>>> These pads are just reserved purpose.
>>> We will plan to remove them in next patch.
>>>
>>> Thanks,
>>>
>>> Jungo
>>>
>>>> Regards,
>>>> Helen
>>>>
>>>>> pad11: Sink
>>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video2
>>>>> pad0: Source
>>>>> -> "mtk-cam-p1":0 []
>>>>>
>>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video3
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video4
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":2 []
>>>>>
>>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video5
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":3 []
>>>>>
>>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video6
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":4 []
>>>>>
>>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video7
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":5 []
>>>>>
>>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video8
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":6 []
>>>>>
>>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>>>              type V4L2 subdev subtype Unknown flags 0
>>>>>              device node name /dev/v4l-subdev1
>>>>> pad0: Sink
>>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>>>> <- "ov8856 2-0010":0 [ENABLED]
>>>>> pad1: Sink
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> <- "ov02a10 4-003d":0 []
>>>>> pad2: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad3: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad4: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>>>> pad5: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad6: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad7: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad8: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad9: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad10: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad11: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>>
>>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev2
>>>>> pad0: Source
>>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>>>> -> "1a040000.seninf":0 [ENABLED]
>>>>>
>>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>>>              type V4L2 subdev subtype Lens flags 0
>>>>>              device node name /dev/v4l-subdev3
>>>>>
>>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev4
>>>>> pad0: Source
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
>>>>> -> "1a040000.seninf":1 []
>>>>>
>>>>> ===========
>>>>> = history =
>>>>> ===========
>>>>>
>>>>> version 6:
>>>>>  - Add port node description in the dt-binding document and device tree
>>>>>    by Tomasz Figa.
>>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>>>    version.
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>>>    timeout event.
>>>>>  - Fix KE due to no sen-inf sub-device.
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>>>  - Add rst documents for Mediatek meta formats.
>>>>>  - New meta buffer structure design & re-factoring.
>>>>>
>>>>> version 5:
>>>>>  - Fixed Rob's comment on dt-binding format
>>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>    and new timestamp type in driver
>>>>>  - Fix buffer en-queue timing issue in v4
>>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 4:
>>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>>>    patch[4]
>>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 3:
>>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>>>    reserved memory region. (Rob Herring)
>>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>>>  - Revise ISP Pass1 Kconfig
>>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>>>    declaration (Hans Verkuil)
>>>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>>>    in-dependently (Hans Verkuil)
>>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Fix ISP de-initialize timing KE issue
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>>>
>>>>> Todo:
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>
>>>>> version 2:
>>>>>  - Add 3A enhancement feature which includes:
>>>>>    Separates 3A pipeline out of frame basis to improve
>>>>>    AE/AWB (exposure and white balance) performance.
>>>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>>>    and co-processor.
>>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>>>  - Revised document wording for dt-bindings documents & dts information.
>>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>>>    or add comments.
>>>>>    Revised buffer size for LMVO & LCSO.
>>>>>    Fix pixel format utility function.
>>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>>>  - mtk_cam-v4l2-util.c
>>>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>>>    Revised stream on function for 3A enhancement feature
>>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Add meta buffer index register definitions
>>>>>    Add meta DMA configuration function.
>>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Add new meta strictures for 3A enhancement feature
>>>>>    Add new IPI command utility function for 3A enhancement feature
>>>>>    Enhance isp_composer_dma_sg_init function flow
>>>>>    Shorten overall IPI command structure size
>>>>>    Remove scp_state state checking
>>>>>    Improve code readability
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>>>    memory region.
>>>>>
>>>>> TODOs:
>>>>>  - 3A enhancement feature bug fixing
>>>>>
>>>>> version 1:
>>>>>  - Revised driver sources based on Tomasz's comments including
>>>>>    part1/2/3/4 in RFC V0 patch.
>>>>>  - Remove DMA cache mechanism.
>>>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>>>    features.
>>>>>  - Fixed v4l2-compliance test failure items.
>>>>>  - Add private controls for Mediatek camera middle-ware.
>>>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>>>    co-processor communication.
>>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>>>    commands RX handling.
>>>>>  - Fix internal bugs.
>>>>>
>>>>> TODOs:
>>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>>>    for shared memory management.
>>>>>  - Revised file names.
>>>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>>>
>>>>> version 0:
>>>>> - Initial submission
>>>>>
>>>>> ==================
>>>>>  Dependent patch set
>>>>> ==================
>>>>>
>>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>>>> The patches are listed as following:
>>>>>
>>>>> [1]. media: support Mediatek sensor interface driver
>>>>> https://patchwork.kernel.org/cover/11145845/
>>>>>
>>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>>>> https://patchwork.kernel.org/cover/11220785/
>>>>>
>>>>> [3]. media: i2c: Add support for OV02A10 sensor
>>>>> https://patchwork.kernel.org/cover/11284779/
>>>>>
>>>>> [4]. media: i2c: add support for DW9768 VCM driver
>>>>> https://patchwork.kernel.org/cover/11132299/
>>>>>
>>>>> [5]. Add support for mt8183 SCP
>>>>> https://patchwork.kernel.org/cover/11239065/
>>>>>
>>>>> [6]. MT8183 IOMMU SUPPORT
>>>>> https://patchwork.kernel.org/cover/11112765/
>>>>>
>>>>> ==================
>>>>>  Compliance test
>>>>> ==================
>>>>>
>>>>> The v4l2-compliance is built with the below lastest patch.
>>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>>>
>>>>> Note 1.
>>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>>>
>>>>> Note 2.
>>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>>>
>>>>> Note 3.
>>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>>>
>>>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>>>
>>>>> v4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media1 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.67
>>>>> Capabilities     : 0x8c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video25 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composiv4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media2 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x8c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video2 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000016
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000014 (20)
>>>>> Name             : mtk-cam-p1 main stream
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000015   : 0: Sink
>>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video3 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300001c
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000001a (26)
>>>>> Name             : mtk-cam-p1 packed out
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100001b   : 0: Sink
>>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video4 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000022
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000020 (32)
>>>>> Name             : mtk-cam-p1 partial meta 0
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000021   : 0: Sink
>>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video5 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000028
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000026 (38)
>>>>> Name             : mtk-cam-p1 partial meta 1
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000027   : 0: Sink
>>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video6 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300002e
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000002c (44)
>>>>> Name             : mtk-cam-p1 partial meta 2
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100002d   : 0: Sink
>>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video7 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000034
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000032 (50)
>>>>> Name             : mtk-cam-p1 partial meta 3
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000033   : 0: Sink
>>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video8 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000050
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000001 (1)
>>>>> Name             : mtk-cam-p1
>>>>> Function         : Video Pixel Formatter
>>>>> Pad 0x01000002   : 0: Sink
>>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>>>> Pad 0x01000003   : 1: Source
>>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>>>> Pad 0x01000004   : 2: Source
>>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>>>> Pad 0x01000005   : 3: Source
>>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>>>> Pad 0x01000006   : 4: Source
>>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>>>> Pad 0x01000007   : 5: Source
>>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>>>> Pad 0x01000008   : 6: Source
>>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>>>> Pad 0x01000009   : 7: Source
>>>>> Pad 0x0100000a   : 8: Source
>>>>> Pad 0x0100000b   : 9: Source
>>>>> Pad 0x0100000c   : 10: Source
>>>>> Pad 0x0100000d   : 11: Sink
>>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev0 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000052
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000038 (56)
>>>>> Name             : 1a040000.seninf
>>>>> Function         : Video Interface Bridge
>>>>> Pad 0x01000039   : 0: Sink
>>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>>>> Pad 0x0100003a   : 1: Sink
>>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>>>> Pad 0x0100003b   : 2: Sink
>>>>> Pad 0x0100003c   : 3: Sink
>>>>> Pad 0x0100003d   : 4: Source
>>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>> Pad 0x0100003e   : 5: Source
>>>>> Pad 0x0100003f   : 6: Source
>>>>> Pad 0x01000040   : 7: Source
>>>>> Pad 0x01000041   : 8: Source
>>>>> Pad 0x01000042   : 9: Source
>>>>> Pad 0x01000043   : 10: Source
>>>>> Pad 0x01000044   : 11: Source
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev1 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000054
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000045 (69)
>>>>> Name             : ov8856 2-0010
>>>>> Function         : Camera Sensor
>>>>> Pad 0x01000046   : 0: Source
>>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev2 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 11 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000056
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000049 (73)
>>>>> Name             : dw9768 2-000c
>>>>> Function         : Lens Controller
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev3 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000058
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x0000004a (74)
>>>>> Name             : ov02a10 4-003d
>>>>> Function         : Camera Sensor
>>>>> Pad 0x0100004b   : 0: Source
>>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev4 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>>>> test VIDIOC_G/S_CTRL: FAIL
>>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 10 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>>>
>>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>>>
>>>>>
>>>>> Jungo Lin (5):
>>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>>>   media: videodev2.h: Add new boottime timestamp type
>>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>>>
>>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>>>  drivers/media/platform/Kconfig                |    1 +
>>>>>  drivers/media/platform/Makefile               |    1 +
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>>>  include/uapi/linux/videodev2.h                |   41 +
>>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-05-05 15:30               ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	ddavenport, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, hverkuil-cisco

Hi Jungo,

On 5/4/20 9:40 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>>
>> Hi Jungo,
>>
>> On 4/10/20 7:32 AM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comment.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hi Jungo,
>>>>
>>>> I was taking a look at this patchset, please see my comments below.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> Hello,
>>>>>
>>>>> This patch series adding the driver for Pass 1 (P1) unit in
>>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>>>> camera features of CrOS.
>>>>>
>>>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>>>> tuning parameters to adjust the image quality. It performs optical
>>>>> black correction, defect pixel correction, W/IR imbalance correction
>>>>> and lens shading correction for RAW processing.
>>>>>
>>>>> The driver is implemented with V4L2 and media controller framework so
>>>>> we have the following entities to describe the ISP pass 1 path.
>>>>>
>>>>> (The current metadata interface used in meta input and partial meta
>>>>> nodes is only a temporary solution to kick off the driver development
>>>>> and is not ready to be reviewed yet.)
>>>>>
>>>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>>>> It accepts the tuning buffer from user.
>>>>>
>>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>>>> main stream and packed out video devices. When processing an image,
>>>>> Pass 1 hardware supports multiple output images with different sizes
>>>>> and formats so it needs two capture video devices ("main stream" and
>>>>> "packed out") to return the image data to the user.
>>>>>
>>>>> 3. main stream (capture video device): return the processed image data
>>>>> which is used in capture scenario.
>>>>>
>>>>> 4. packed out (capture video device): return the processed image data
>>>>> which is used in preview scenario.
>>>>>
>>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>>>
>>>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>>>
>>>>> 7. partial meta 2 (capture video device): return the local contrast
>>>>>    enhanced statistics.
>>>>>
>>>>> 8. partial meta 3 (capture video device): return the local motion
>>>>>    vector statistics.
>>>>>
>>>>> The overall patches of the series is:
>>>>>
>>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>>>   communication to compose ISP registers in the firmware.
>>>>>
>>>>> Here is ISP P1 media topology:
>>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>>>> which are implemented in below patch[1][2][3][4]:
>>>>
>>>> I would be nice if you could provide a branch with those applied.
>>>>
>>>
>>> We apply those patches in the chromeos-4.19 to test.
>>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
>>>
>>>
>>>>>
>>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>>>> patch sets.
>>>>>
>>>>> /usr/bin/media-ctl -p -d /dev/media2
>>>>>
>>>>> Media controller API version 4.19.89
>>>>>
>>>>> Media device information
>>>>> ------------------------
>>>>> driver          mtk-cam-p1
>>>>> model           mtk-cam-p1
>>>>> serial
>>>>> bus info        platform:1a000000.camisp
>>>>> hw revision     0x0
>>>>> driver version  4.19.89
>>>>>
>>>>> Device topology
>>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>>>
>>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>>>> Is this correct?
>>>>
>>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>>>
>>>
>>> Currently, we only support single-cam in this SoC with upstream driver.
>>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
>>> we'd like to keep the naming to avoid confusion.
>>
>> I suppose this new Mediatek SoC would use this same driver?
>> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
>> naming will be different then, right? (I guess it's ok).
>>
> 
> Sorry, my last comment should be corrected.
> The new Mediatek SoC with new ISP HW version will support multi-cam
> capabilities with new upstream driver. Due to the new enrich function,
> it may not reuse current driver.

right, thanks for the clarification.

> 
>>>
>>>>>             type V4L2 subdev subtype Unknown flags 0
>>>>>             device node name /dev/v4l-subdev0
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1 meta input":0 []
>>>>
>>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>>>
>>>
>>> Ok, we will revise our naming in next patch.
>>>
>>>>> pad1: Source
>>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>>>
>>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>>>
>>>
>>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
>>>
>>>>> pad2: Source
>>>>> -> "mtk-cam-p1 packed out":0 []
>>>>
>>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>>>
>>>
>>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
>>> padding in the memory. We may revise the naming in next patch.
>>>
>>>>> pad3: Source
>>>>> -> "mtk-cam-p1 partial meta 0":0 []
>>>>> pad4: Source
>>>>> -> "mtk-cam-p1 partial meta 1":0 []
>>>>> pad5: Source
>>>>> -> "mtk-cam-p1 partial meta 2":0 []
>>>>> pad6: Source
>>>>> -> "mtk-cam-p1 partial meta 3":0 []
>>>>
>>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>>>
>>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>>>
>>>> I also would prefer to remove blank spaces.
>>>>
>>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>>>
>>>
>>> No, these links are optional to setup for userspace.
>>
>> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
>> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
>>
>> I'm not sure if using the link state is the best option (please see my comment on 5/5).
>> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
>> interesting nodes at once.
>>
>>
>> Regards,
>> Helen
>>
> 
> 
> According to your suggestion, do you have sample code about "userspace
> could do a request to enable the stream in all the interesting nodes at
> once"? If this supports, it is helpful for us to simply our current
> implementation.

I was checking the request api docs [1] in more details and it seems this isn't possible, since
"A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
on MEDIA_IOC_REQUEST_ALLOC doc[2].

[1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
[2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc

What bothers me with the current implementation is that users could correctly configure the topology, but when
calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
for userspace about what is wrong.

So I was trying to think about how this could be done differently.
I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

Regards,
Helen


> 
> 
> Thanks,
> 
> 
> Jungo
> 
> 
>>> For naming part, we will align with driver source codes.
>>>
>>>>> pad7: Source
>>>>> pad8: Source
>>>>> pad9: Source
>>>>> pad10: Source
>>>>
>>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>>>
>>>
>>> These pads are just reserved purpose.
>>> We will plan to remove them in next patch.
>>>
>>> Thanks,
>>>
>>> Jungo
>>>
>>>> Regards,
>>>> Helen
>>>>
>>>>> pad11: Sink
>>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video2
>>>>> pad0: Source
>>>>> -> "mtk-cam-p1":0 []
>>>>>
>>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video3
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video4
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":2 []
>>>>>
>>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video5
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":3 []
>>>>>
>>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video6
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":4 []
>>>>>
>>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video7
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":5 []
>>>>>
>>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video8
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":6 []
>>>>>
>>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>>>              type V4L2 subdev subtype Unknown flags 0
>>>>>              device node name /dev/v4l-subdev1
>>>>> pad0: Sink
>>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>>>> <- "ov8856 2-0010":0 [ENABLED]
>>>>> pad1: Sink
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> <- "ov02a10 4-003d":0 []
>>>>> pad2: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad3: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad4: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>>>> pad5: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad6: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad7: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad8: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad9: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad10: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad11: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>>
>>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev2
>>>>> pad0: Source
>>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>>>> -> "1a040000.seninf":0 [ENABLED]
>>>>>
>>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>>>              type V4L2 subdev subtype Lens flags 0
>>>>>              device node name /dev/v4l-subdev3
>>>>>
>>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev4
>>>>> pad0: Source
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
>>>>> -> "1a040000.seninf":1 []
>>>>>
>>>>> ===========
>>>>> = history =
>>>>> ===========
>>>>>
>>>>> version 6:
>>>>>  - Add port node description in the dt-binding document and device tree
>>>>>    by Tomasz Figa.
>>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>>>    version.
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>>>    timeout event.
>>>>>  - Fix KE due to no sen-inf sub-device.
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>>>  - Add rst documents for Mediatek meta formats.
>>>>>  - New meta buffer structure design & re-factoring.
>>>>>
>>>>> version 5:
>>>>>  - Fixed Rob's comment on dt-binding format
>>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>    and new timestamp type in driver
>>>>>  - Fix buffer en-queue timing issue in v4
>>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 4:
>>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>>>    patch[4]
>>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 3:
>>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>>>    reserved memory region. (Rob Herring)
>>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>>>  - Revise ISP Pass1 Kconfig
>>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>>>    declaration (Hans Verkuil)
>>>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>>>    in-dependently (Hans Verkuil)
>>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Fix ISP de-initialize timing KE issue
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>>>
>>>>> Todo:
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>
>>>>> version 2:
>>>>>  - Add 3A enhancement feature which includes:
>>>>>    Separates 3A pipeline out of frame basis to improve
>>>>>    AE/AWB (exposure and white balance) performance.
>>>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>>>    and co-processor.
>>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>>>  - Revised document wording for dt-bindings documents & dts information.
>>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>>>    or add comments.
>>>>>    Revised buffer size for LMVO & LCSO.
>>>>>    Fix pixel format utility function.
>>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>>>  - mtk_cam-v4l2-util.c
>>>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>>>    Revised stream on function for 3A enhancement feature
>>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Add meta buffer index register definitions
>>>>>    Add meta DMA configuration function.
>>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Add new meta strictures for 3A enhancement feature
>>>>>    Add new IPI command utility function for 3A enhancement feature
>>>>>    Enhance isp_composer_dma_sg_init function flow
>>>>>    Shorten overall IPI command structure size
>>>>>    Remove scp_state state checking
>>>>>    Improve code readability
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>>>    memory region.
>>>>>
>>>>> TODOs:
>>>>>  - 3A enhancement feature bug fixing
>>>>>
>>>>> version 1:
>>>>>  - Revised driver sources based on Tomasz's comments including
>>>>>    part1/2/3/4 in RFC V0 patch.
>>>>>  - Remove DMA cache mechanism.
>>>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>>>    features.
>>>>>  - Fixed v4l2-compliance test failure items.
>>>>>  - Add private controls for Mediatek camera middle-ware.
>>>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>>>    co-processor communication.
>>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>>>    commands RX handling.
>>>>>  - Fix internal bugs.
>>>>>
>>>>> TODOs:
>>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>>>    for shared memory management.
>>>>>  - Revised file names.
>>>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>>>
>>>>> version 0:
>>>>> - Initial submission
>>>>>
>>>>> ==================
>>>>>  Dependent patch set
>>>>> ==================
>>>>>
>>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>>>> The patches are listed as following:
>>>>>
>>>>> [1]. media: support Mediatek sensor interface driver
>>>>> https://patchwork.kernel.org/cover/11145845/
>>>>>
>>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>>>> https://patchwork.kernel.org/cover/11220785/
>>>>>
>>>>> [3]. media: i2c: Add support for OV02A10 sensor
>>>>> https://patchwork.kernel.org/cover/11284779/
>>>>>
>>>>> [4]. media: i2c: add support for DW9768 VCM driver
>>>>> https://patchwork.kernel.org/cover/11132299/
>>>>>
>>>>> [5]. Add support for mt8183 SCP
>>>>> https://patchwork.kernel.org/cover/11239065/
>>>>>
>>>>> [6]. MT8183 IOMMU SUPPORT
>>>>> https://patchwork.kernel.org/cover/11112765/
>>>>>
>>>>> ==================
>>>>>  Compliance test
>>>>> ==================
>>>>>
>>>>> The v4l2-compliance is built with the below lastest patch.
>>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>>>
>>>>> Note 1.
>>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>>>
>>>>> Note 2.
>>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>>>
>>>>> Note 3.
>>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>>>
>>>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>>>
>>>>> v4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media1 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.67
>>>>> Capabilities     : 0x8c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video25 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composiv4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media2 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x8c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video2 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000016
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000014 (20)
>>>>> Name             : mtk-cam-p1 main stream
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000015   : 0: Sink
>>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video3 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300001c
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000001a (26)
>>>>> Name             : mtk-cam-p1 packed out
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100001b   : 0: Sink
>>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video4 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000022
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000020 (32)
>>>>> Name             : mtk-cam-p1 partial meta 0
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000021   : 0: Sink
>>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video5 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000028
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000026 (38)
>>>>> Name             : mtk-cam-p1 partial meta 1
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000027   : 0: Sink
>>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video6 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300002e
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000002c (44)
>>>>> Name             : mtk-cam-p1 partial meta 2
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100002d   : 0: Sink
>>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video7 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000034
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000032 (50)
>>>>> Name             : mtk-cam-p1 partial meta 3
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000033   : 0: Sink
>>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video8 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000050
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000001 (1)
>>>>> Name             : mtk-cam-p1
>>>>> Function         : Video Pixel Formatter
>>>>> Pad 0x01000002   : 0: Sink
>>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>>>> Pad 0x01000003   : 1: Source
>>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>>>> Pad 0x01000004   : 2: Source
>>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>>>> Pad 0x01000005   : 3: Source
>>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>>>> Pad 0x01000006   : 4: Source
>>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>>>> Pad 0x01000007   : 5: Source
>>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>>>> Pad 0x01000008   : 6: Source
>>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>>>> Pad 0x01000009   : 7: Source
>>>>> Pad 0x0100000a   : 8: Source
>>>>> Pad 0x0100000b   : 9: Source
>>>>> Pad 0x0100000c   : 10: Source
>>>>> Pad 0x0100000d   : 11: Sink
>>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev0 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000052
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000038 (56)
>>>>> Name             : 1a040000.seninf
>>>>> Function         : Video Interface Bridge
>>>>> Pad 0x01000039   : 0: Sink
>>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>>>> Pad 0x0100003a   : 1: Sink
>>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>>>> Pad 0x0100003b   : 2: Sink
>>>>> Pad 0x0100003c   : 3: Sink
>>>>> Pad 0x0100003d   : 4: Source
>>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>> Pad 0x0100003e   : 5: Source
>>>>> Pad 0x0100003f   : 6: Source
>>>>> Pad 0x01000040   : 7: Source
>>>>> Pad 0x01000041   : 8: Source
>>>>> Pad 0x01000042   : 9: Source
>>>>> Pad 0x01000043   : 10: Source
>>>>> Pad 0x01000044   : 11: Source
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev1 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000054
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000045 (69)
>>>>> Name             : ov8856 2-0010
>>>>> Function         : Camera Sensor
>>>>> Pad 0x01000046   : 0: Source
>>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev2 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 11 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000056
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000049 (73)
>>>>> Name             : dw9768 2-000c
>>>>> Function         : Lens Controller
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev3 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000058
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x0000004a (74)
>>>>> Name             : ov02a10 4-003d
>>>>> Function         : Camera Sensor
>>>>> Pad 0x0100004b   : 0: Source
>>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev4 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>>>> test VIDIOC_G/S_CTRL: FAIL
>>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 10 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>>>
>>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>>>
>>>>>
>>>>> Jungo Lin (5):
>>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>>>   media: videodev2.h: Add new boottime timestamp type
>>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>>>
>>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>>>  drivers/media/platform/Kconfig                |    1 +
>>>>>  drivers/media/platform/Makefile               |    1 +
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>>>  include/uapi/linux/videodev2.h                |   41 +
>>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-05-05 15:30               ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:30 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	ddavenport, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, hverkuil-cisco

Hi Jungo,

On 5/4/20 9:40 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>>
>> Hi Jungo,
>>
>> On 4/10/20 7:32 AM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comment.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hi Jungo,
>>>>
>>>> I was taking a look at this patchset, please see my comments below.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> Hello,
>>>>>
>>>>> This patch series adding the driver for Pass 1 (P1) unit in
>>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
>>>>> camera features of CrOS.
>>>>>
>>>>> Pass 1 unit processes image signal from sensor devices and accepts the
>>>>> tuning parameters to adjust the image quality. It performs optical
>>>>> black correction, defect pixel correction, W/IR imbalance correction
>>>>> and lens shading correction for RAW processing.
>>>>>
>>>>> The driver is implemented with V4L2 and media controller framework so
>>>>> we have the following entities to describe the ISP pass 1 path.
>>>>>
>>>>> (The current metadata interface used in meta input and partial meta
>>>>> nodes is only a temporary solution to kick off the driver development
>>>>> and is not ready to be reviewed yet.)
>>>>>
>>>>> 1. meta input (output video device): connect to ISP P1 sub device.
>>>>> It accepts the tuning buffer from user.
>>>>>
>>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
>>>>> main stream and packed out video devices. When processing an image,
>>>>> Pass 1 hardware supports multiple output images with different sizes
>>>>> and formats so it needs two capture video devices ("main stream" and
>>>>> "packed out") to return the image data to the user.
>>>>>
>>>>> 3. main stream (capture video device): return the processed image data
>>>>> which is used in capture scenario.
>>>>>
>>>>> 4. packed out (capture video device): return the processed image data
>>>>> which is used in preview scenario.
>>>>>
>>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
>>>>>
>>>>> 6. partial meta 1 (capture video device): return the AF statistics.
>>>>>
>>>>> 7. partial meta 2 (capture video device): return the local contrast
>>>>>    enhanced statistics.
>>>>>
>>>>> 8. partial meta 3 (capture video device): return the local motion
>>>>>    vector statistics.
>>>>>
>>>>> The overall patches of the series is:
>>>>>
>>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
>>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
>>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
>>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
>>>>>   Moreover, implement standard V4L2 video driver that utilizes
>>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
>>>>>   communication to compose ISP registers in the firmware.
>>>>>
>>>>> Here is ISP P1 media topology:
>>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
>>>>> which are implemented in below patch[1][2][3][4]:
>>>>
>>>> I would be nice if you could provide a branch with those applied.
>>>>
>>>
>>> We apply those patches in the chromeos-4.19 to test.
>>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
>>>
>>>
>>>>>
>>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
>>>>> patch sets.
>>>>>
>>>>> /usr/bin/media-ctl -p -d /dev/media2
>>>>>
>>>>> Media controller API version 4.19.89
>>>>>
>>>>> Media device information
>>>>> ------------------------
>>>>> driver          mtk-cam-p1
>>>>> model           mtk-cam-p1
>>>>> serial
>>>>> bus info        platform:1a000000.camisp
>>>>> hw revision     0x0
>>>>> driver version  4.19.89
>>>>>
>>>>> Device topology
>>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
>>>>
>>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
>>>> Is this correct?
>>>>
>>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
>>>>
>>>
>>> Currently, we only support single-cam in this SoC with upstream driver.
>>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
>>> we'd like to keep the naming to avoid confusion.
>>
>> I suppose this new Mediatek SoC would use this same driver?
>> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
>> naming will be different then, right? (I guess it's ok).
>>
> 
> Sorry, my last comment should be corrected.
> The new Mediatek SoC with new ISP HW version will support multi-cam
> capabilities with new upstream driver. Due to the new enrich function,
> it may not reuse current driver.

right, thanks for the clarification.

> 
>>>
>>>>>             type V4L2 subdev subtype Unknown flags 0
>>>>>             device node name /dev/v4l-subdev0
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1 meta input":0 []
>>>>
>>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
>>>>
>>>
>>> Ok, we will revise our naming in next patch.
>>>
>>>>> pad1: Source
>>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
>>>>
>>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
>>>>
>>>
>>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
>>>
>>>>> pad2: Source
>>>>> -> "mtk-cam-p1 packed out":0 []
>>>>
>>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
>>>>
>>>
>>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
>>> padding in the memory. We may revise the naming in next patch.
>>>
>>>>> pad3: Source
>>>>> -> "mtk-cam-p1 partial meta 0":0 []
>>>>> pad4: Source
>>>>> -> "mtk-cam-p1 partial meta 1":0 []
>>>>> pad5: Source
>>>>> -> "mtk-cam-p1 partial meta 2":0 []
>>>>> pad6: Source
>>>>> -> "mtk-cam-p1 partial meta 3":0 []
>>>>
>>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
>>>>
>>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
>>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
>>>>
>>>> I also would prefer to remove blank spaces.
>>>>
>>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
>>>>
>>>
>>> No, these links are optional to setup for userspace.
>>
>> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
>> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
>>
>> I'm not sure if using the link state is the best option (please see my comment on 5/5).
>> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
>> interesting nodes at once.
>>
>>
>> Regards,
>> Helen
>>
> 
> 
> According to your suggestion, do you have sample code about "userspace
> could do a request to enable the stream in all the interesting nodes at
> once"? If this supports, it is helpful for us to simply our current
> implementation.

I was checking the request api docs [1] in more details and it seems this isn't possible, since
"A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
on MEDIA_IOC_REQUEST_ALLOC doc[2].

[1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
[2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc

What bothers me with the current implementation is that users could correctly configure the topology, but when
calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
for userspace about what is wrong.

So I was trying to think about how this could be done differently.
I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

Regards,
Helen


> 
> 
> Thanks,
> 
> 
> Jungo
> 
> 
>>> For naming part, we will align with driver source codes.
>>>
>>>>> pad7: Source
>>>>> pad8: Source
>>>>> pad9: Source
>>>>> pad10: Source
>>>>
>>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
>>>>
>>>
>>> These pads are just reserved purpose.
>>> We will plan to remove them in next patch.
>>>
>>> Thanks,
>>>
>>> Jungo
>>>
>>>> Regards,
>>>> Helen
>>>>
>>>>> pad11: Sink
>>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video2
>>>>> pad0: Source
>>>>> -> "mtk-cam-p1":0 []
>>>>>
>>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video3
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
>>>>>
>>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video4
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":2 []
>>>>>
>>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video5
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":3 []
>>>>>
>>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video6
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":4 []
>>>>>
>>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video7
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":5 []
>>>>>
>>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
>>>>>              type Node subtype V4L flags 0
>>>>>              device node name /dev/video8
>>>>> pad0: Sink
>>>>> <- "mtk-cam-p1":6 []
>>>>>
>>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
>>>>>              type V4L2 subdev subtype Unknown flags 0
>>>>>              device node name /dev/v4l-subdev1
>>>>> pad0: Sink
>>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
>>>>> <- "ov8856 2-0010":0 [ENABLED]
>>>>> pad1: Sink
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> <- "ov02a10 4-003d":0 []
>>>>> pad2: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad3: Sink
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad4: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
>>>>> pad5: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad6: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad7: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad8: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad9: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad10: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>> pad11: Source
>>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
>>>>>
>>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev2
>>>>> pad0: Source
>>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
>>>>> -> "1a040000.seninf":0 [ENABLED]
>>>>>
>>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
>>>>>              type V4L2 subdev subtype Lens flags 0
>>>>>              device node name /dev/v4l-subdev3
>>>>>
>>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
>>>>>              type V4L2 subdev subtype Sensor flags 0
>>>>>              device node name /dev/v4l-subdev4
>>>>> pad0: Source
>>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
>>>>> -> "1a040000.seninf":1 []
>>>>>
>>>>> ===========
>>>>> = history =
>>>>> ===========
>>>>>
>>>>> version 6:
>>>>>  - Add port node description in the dt-binding document and device tree
>>>>>    by Tomasz Figa.
>>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
>>>>>    version.
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
>>>>>  - Correct auto suspend timer value for suspend/resume issue.
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
>>>>>    timeout event.
>>>>>  - Fix KE due to no sen-inf sub-device.
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation.
>>>>>  - Add rst documents for Mediatek meta formats.
>>>>>  - New meta buffer structure design & re-factoring.
>>>>>
>>>>> version 5:
>>>>>  - Fixed Rob's comment on dt-binding format
>>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>    and new timestamp type in driver
>>>>>  - Fix buffer en-queue timing issue in v4
>>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 4:
>>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
>>>>>    patch[4]
>>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
>>>>>  - Extend Mediatek proprietary image formats to support bayer order
>>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
>>>>>
>>>>> Todo:
>>>>>  - vb2_ops's buf_request_complete callback function implementation
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
>>>>>  - Align and pack IPI command structures for EC ROM size shrink
>>>>>
>>>>> version 3:
>>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
>>>>>    reserved memory region. (Rob Herring)
>>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
>>>>>  - Revise ISP Pass1 Kconfig
>>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
>>>>>  - Fix kernel warning messages when running v4l2_compliance test
>>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
>>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
>>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
>>>>>    declaration (Hans Verkuil)
>>>>>    Split GET_BIN_INFO control into two controls to get width & height
>>>>>    in-dependently (Hans Verkuil)
>>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
>>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
>>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Fix Drew's comments which are addressed in v2 patch
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
>>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
>>>>>    Fix ISP de-initialize timing KE issue
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
>>>>>
>>>>> Todo:
>>>>>  - Add rst documents for Mediatek meta formats
>>>>>  - New meta buffer structure design & re-factoring
>>>>>
>>>>> version 2:
>>>>>  - Add 3A enhancement feature which includes:
>>>>>    Separates 3A pipeline out of frame basis to improve
>>>>>    AE/AWB (exposure and white balance) performance.
>>>>>    Add 2 SCP sub-commands for 3A meta buffers.
>>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
>>>>>    and co-processor.
>>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
>>>>>  - Revised document wording for dt-bindings documents & dts information.
>>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
>>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
>>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
>>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
>>>>>    or add comments.
>>>>>    Revised buffer size for LMVO & LCSO.
>>>>>    Fix pixel format utility function.
>>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
>>>>>  - mtk_cam-v4l2-util.c
>>>>>    Refactoring V4L2 async mechanism with seninf driver only
>>>>>    Refactoring CIO (Connection IO) implementation with active sensor
>>>>>    Revised stream on function for 3A enhancement feature
>>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
>>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
>>>>>    Add meta buffer index register definitions
>>>>>    Add meta DMA configuration function.
>>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
>>>>>    Add isp_setup_scp_rproc function to get RPC handle
>>>>>    Add mtk_cam_reserved_memory_init for shared memory management
>>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
>>>>>    Add new meta strictures for 3A enhancement feature
>>>>>    Add new IPI command utility function for 3A enhancement feature
>>>>>    Enhance isp_composer_dma_sg_init function flow
>>>>>    Shorten overall IPI command structure size
>>>>>    Remove scp_state state checking
>>>>>    Improve code readability
>>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
>>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
>>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
>>>>>    memory region.
>>>>>
>>>>> TODOs:
>>>>>  - 3A enhancement feature bug fixing
>>>>>
>>>>> version 1:
>>>>>  - Revised driver sources based on Tomasz's comments including
>>>>>    part1/2/3/4 in RFC V0 patch.
>>>>>  - Remove DMA cache mechanism.
>>>>>    Support two new video devices (LCSO/LMVO) for advance camera
>>>>>    features.
>>>>>  - Fixed v4l2-compliance test failure items.
>>>>>  - Add private controls for Mediatek camera middle-ware.
>>>>>  - Replace VPU driver's APIs with new SCP driver interface for
>>>>>    co-processor communication.
>>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
>>>>>    commands RX handling.
>>>>>  - Fix internal bugs.
>>>>>
>>>>> TODOs:
>>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
>>>>>    for shared memory management.
>>>>>  - Revised file names.
>>>>>  - Support non frame-sync AFO/AAO DMA buffers
>>>>>
>>>>> version 0:
>>>>> - Initial submission
>>>>>
>>>>> ==================
>>>>>  Dependent patch set
>>>>> ==================
>>>>>
>>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
>>>>> The patches are listed as following:
>>>>>
>>>>> [1]. media: support Mediatek sensor interface driver
>>>>> https://patchwork.kernel.org/cover/11145845/
>>>>>
>>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
>>>>> https://patchwork.kernel.org/cover/11220785/
>>>>>
>>>>> [3]. media: i2c: Add support for OV02A10 sensor
>>>>> https://patchwork.kernel.org/cover/11284779/
>>>>>
>>>>> [4]. media: i2c: add support for DW9768 VCM driver
>>>>> https://patchwork.kernel.org/cover/11132299/
>>>>>
>>>>> [5]. Add support for mt8183 SCP
>>>>> https://patchwork.kernel.org/cover/11239065/
>>>>>
>>>>> [6]. MT8183 IOMMU SUPPORT
>>>>> https://patchwork.kernel.org/cover/11112765/
>>>>>
>>>>> ==================
>>>>>  Compliance test
>>>>> ==================
>>>>>
>>>>> The v4l2-compliance is built with the below lastest patch.
>>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
>>>>>
>>>>> Note 1.
>>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
>>>>>
>>>>> Note 2.
>>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
>>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
>>>>>
>>>>> Note 3.
>>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
>>>>>
>>>>> /usr/bin/v4l2-compliance -m /dev/media2
>>>>>
>>>>> v4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media1 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video25:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.67
>>>>> Capabilities     : 0x8c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.67
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.67
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video25 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composiv4l2-compliance SHA: not available, 32 bits
>>>>>
>>>>> Compliance test for mtk-cam-p1 device /dev/media2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>>
>>>>> Required ioctls:
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/media2 open: OK
>>>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Media Controller ioctls:
>>>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
>>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>>>> test MEDIA_IOC_SETUP_LINK: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video2:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x8c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x0c200000
>>>>> Metadata Output
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000010
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000000e (14)
>>>>> Name             : mtk-cam-p1 meta input
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100000f   : 0: Source
>>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video2 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video3:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000016
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000014 (20)
>>>>> Name             : mtk-cam-p1 main stream
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000015   : 0: Sink
>>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video3 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video4:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04201000
>>>>> Video Capture Multiplanar
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300001c
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000001a (26)
>>>>> Name             : mtk-cam-p1 packed out
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100001b   : 0: Sink
>>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video4 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video5:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000022
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000020 (32)
>>>>> Name             : mtk-cam-p1 partial meta 0
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000021   : 0: Sink
>>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video5 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video6:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000028
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000026 (38)
>>>>> Name             : mtk-cam-p1 partial meta 1
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000027   : 0: Sink
>>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video6 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video7:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x0300002e
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x0000002c (44)
>>>>> Name             : mtk-cam-p1 partial meta 2
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x0100002d   : 0: Sink
>>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video7 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/video8:
>>>>>
>>>>> Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Card type        : mtk-cam-p1
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Driver version   : 4.19.89
>>>>> Capabilities     : 0x84a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Device Capabilities
>>>>> Device Caps      : 0x04a00000
>>>>> Metadata Capture
>>>>> Streaming
>>>>> Extended Pix Format
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000034
>>>>> Type             : V4L Video
>>>>> Entity Info:
>>>>> ID               : 0x00000032 (50)
>>>>> Name             : mtk-cam-p1 partial meta 3
>>>>> Function         : V4L2 I/O
>>>>> Pad 0x01000033   : 0: Sink
>>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/video8 open: OK
>>>>> test VIDIOC_QUERYCAP: OK
>>>>> test VIDIOC_G/S_PRIORITY: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK
>>>>> test VIDIOC_TRY_FMT: OK
>>>>> test VIDIOC_S_FMT: OK
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
>>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
>>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
>>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000050
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000001 (1)
>>>>> Name             : mtk-cam-p1
>>>>> Function         : Video Pixel Formatter
>>>>> Pad 0x01000002   : 0: Sink
>>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
>>>>> Pad 0x01000003   : 1: Source
>>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
>>>>> Pad 0x01000004   : 2: Source
>>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
>>>>> Pad 0x01000005   : 3: Source
>>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
>>>>> Pad 0x01000006   : 4: Source
>>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
>>>>> Pad 0x01000007   : 5: Source
>>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
>>>>> Pad 0x01000008   : 6: Source
>>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
>>>>> Pad 0x01000009   : 7: Source
>>>>> Pad 0x0100000a   : 8: Source
>>>>> Pad 0x0100000b   : 9: Source
>>>>> Pad 0x0100000c   : 10: Source
>>>>> Pad 0x0100000d   : 11: Sink
>>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev0 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 0 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000052
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000038 (56)
>>>>> Name             : 1a040000.seninf
>>>>> Function         : Video Interface Bridge
>>>>> Pad 0x01000039   : 0: Sink
>>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
>>>>> Pad 0x0100003a   : 1: Sink
>>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
>>>>> Pad 0x0100003b   : 2: Sink
>>>>> Pad 0x0100003c   : 3: Sink
>>>>> Pad 0x0100003d   : 4: Source
>>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
>>>>> Pad 0x0100003e   : 5: Source
>>>>> Pad 0x0100003f   : 6: Source
>>>>> Pad 0x01000040   : 7: Source
>>>>> Pad 0x01000041   : 8: Source
>>>>> Pad 0x01000042   : 9: Source
>>>>> Pad 0x01000043   : 10: Source
>>>>> Pad 0x01000044   : 11: Source
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev1 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 1):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 2):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Sink Pad 3):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 4):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 5):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 6):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 7):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 8):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 9):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 10):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 11):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000054
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000045 (69)
>>>>> Name             : ov8856 2-0010
>>>>> Function         : Camera Sensor
>>>>> Pad 0x01000046   : 0: Source
>>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev2 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
>>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
>>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 11 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000056
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x00000049 (73)
>>>>> Name             : dw9768 2-000c
>>>>> Function         : Lens Controller
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev3 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> test VIDIOC_G/S_CTRL: OK
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 2 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
>>>>> --------------------------------------------------------------------------------
>>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
>>>>>
>>>>> Media Driver Info:
>>>>> Driver name      : mtk-cam-p1
>>>>> Model            : mtk-cam-p1
>>>>> Serial           :
>>>>> Bus info         : platform:1a000000.camisp
>>>>> Media version    : 4.19.89
>>>>> Hardware revision: 0x00000000 (0)
>>>>> Driver version   : 4.19.89
>>>>> Interface Info:
>>>>> ID               : 0x03000058
>>>>> Type             : V4L Sub-Device
>>>>> Entity Info:
>>>>> ID               : 0x0000004a (74)
>>>>> Name             : ov02a10 4-003d
>>>>> Function         : Camera Sensor
>>>>> Pad 0x0100004b   : 0: Source
>>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
>>>>>
>>>>> Required ioctls:
>>>>> test MC information (see 'Media Driver Info' above): OK
>>>>>
>>>>> Allow for multiple opens:
>>>>> test second /dev/v4l-subdev4 open: OK
>>>>> test for unlimited opens: OK
>>>>>
>>>>> Debug ioctls:
>>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>>
>>>>> Input ioctls:
>>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>>
>>>>> Output ioctls:
>>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>>
>>>>> Input/Output configuration ioctls:
>>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>>
>>>>> Sub-Device ioctls (Source Pad 0):
>>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
>>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
>>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
>>>>>
>>>>> Control ioctls:
>>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>>> test VIDIOC_QUERYCTRL: OK
>>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
>>>>> test VIDIOC_G/S_CTRL: FAIL
>>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
>>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
>>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
>>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
>>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>>> Standard Controls: 10 Private Controls: 0
>>>>>
>>>>> Format ioctls:
>>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
>>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>>> test VIDIOC_G_FMT: OK (Not Supported)
>>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
>>>>> test VIDIOC_S_FMT: OK (Not Supported)
>>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>>> test Cropping: OK (Not Supported)
>>>>> test Composing: OK (Not Supported)
>>>>> test Scaling: OK (Not Supported)
>>>>>
>>>>> Codec ioctls:
>>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>>
>>>>> Buffer ioctls:
>>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
>>>>> test VIDIOC_EXPBUF: OK (Not Supported)
>>>>> test Requests: OK (Not Supported)
>>>>>
>>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
>>>>>
>>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
>>>>>
>>>>>
>>>>> Jungo Lin (5):
>>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
>>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
>>>>>   media: videodev2.h: Add new boottime timestamp type
>>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
>>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
>>>>>
>>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
>>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
>>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
>>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
>>>>>  drivers/media/platform/Kconfig                |    1 +
>>>>>  drivers/media/platform/Makefile               |    1 +
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
>>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
>>>>>  include/uapi/linux/videodev2.h                |   41 +
>>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
>>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
>>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
>>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> 
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
  2020-05-04 12:27               ` Jungo Lin
  (?)
@ 2020-05-05 15:38                 ` Helen Koike
  -1 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:38 UTC (permalink / raw)
  To: Jungo Lin
  Cc: laurent.pinchart, matthias.bgg, mchehab, shik, devicetree,
	Sean.Cheng, suleiman, Pi-Hsun Shih, srv_heupstream, robh,
	ryan.yu, Jerry-ch.Chen, frankie.chiu, sj.huang, yuzhao,
	linux-mediatek, zwisler, ddavenport, frederic.chen,
	linux-arm-kernel, linux-media, tfiga, hverkuil-cisco



On 5/4/20 9:27 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>> On 4/8/20 11:05 PM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comments.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hello Jungo,
>>>>
>>>> I was taking a look at this patch (thanks for the work),
>>>> I didn't look in deep details, but I have some comments, please see
>>>> below. I hope it helps.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> This patch adds the Mediatek ISP P1 HW control device driver.
>>>>> It handles the ISP HW configuration, provides interrupt handling and
>>>>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
>>>>> implement standard V4L2 video driver that utilizes V4L2 and media
>>>>> framework APIs. It supports one media device, one sub-device and
>>>>> several video devices during initialization. Moreover, it also connects
>>>>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
>>>>> co-process via SCP communication to compose ISP registers in the
>>>>> firmware.
>>>>>
>>>>> (The current metadata interface used in meta input and partial
>>>>> meta nodes is only a temporary solution to kick off the driver
>>>>> development and is not ready to be reviewed yet.)
>>>>>
>>>>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
>>>>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
>>>>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
>>>>> ---
>>>>> Changes from v6:
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>>>>>  - Correct auto suspend timer value for suspend/resume issue
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>>>>>  - Fix KE due to no sen-inf sub-device
>>>>> ---
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>
>>>> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
>>>> capture, which for metadata, and what is being used to deal with requests.
>>>>
>>>> It would make it easier to review imho.
>>>>
>>>
>>> For file structure design, it was reviewed in the previous patch
>>> serials.
>>> e.g.
>>> https://patchwork.kernel.org/patch/10938137/
>>> If you think it is better, I will modify it.
>>
>> Right, I saw a suggestion to merge two files there.
>>
>> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
>> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
>> So it reflects a bit the topology.
>> But it is also up to you to see if it improves organization or not, it is just a suggestion.
>>
> 
> Ok, got your point.
> We will discuss how to do internally.
> 
> [snip]
>>>>> +isp_composer_hw_init(p1_dev);
>>>>> +
>>>>> +p1_dev->enqueued_frame_seq_no = 0;
>>>>> +p1_dev->dequeued_frame_seq_no = 0;
>>>>> +p1_dev->composed_frame_seq_no = 0;
>>>>> +p1_dev->sof_count = 0;
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>>>> +
>>>>> +isp_composer_hw_deinit(p1_dev);
>>>>> +pm_runtime_mark_last_busy(dev);
>>>>> +pm_runtime_put_autosuspend(dev);
>>>>> +rproc_shutdown(p1_dev->rproc_handle);
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>>>>> + struct mtk_cam_dev_request *req)
>>>>> +{
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>>>> +
>>>>> +/* Accumulated frame sequence number */
>>>>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
>>>>> +
>>>>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>>>>> +queue_work(p1_dev->composer_wq, &req->frame_work);
>>>>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no,
>>>>> +cam->running_job_count);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>>>>> +       unsigned int dequeued_frame_seq_no)
>>>>> +{
>>>>> +dma_addr_t base_addr = p1_dev->composer_iova;
>>>>> +struct device *dev = p1_dev->dev;
>>>>> +struct mtk_cam_dev_request *req;
>>>>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
>>>>> +unsigned int addr_offset;
>>>>> +
>>>>> +/* Send V4L2_EVENT_FRAME_SYNC event */
>>>>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +
>>>>> +p1_dev->sof_count += 1;
>>>>> +/* Save frame information */
>>>>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
>>>>> +
>>>>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +if (req)
>>>>> +req->timestamp = ktime_get_boottime_ns();
>>>>> +
>>>>> +/* Update CQ base address if needed */
>>>>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no);
>>>>> +return;
>>>>> +}
>>>>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>>>>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>>>>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
>>>>> +{
>>>>> +u32 val;
>>>>> +
>>>>> +dev_err(p1_dev->dev,
>>>>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
>>>>> +dev_err(p1_dev->dev,
>>>>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
>>>>
>>>> I think if would be better to transfor those into dev_dbg and add a counter
>>>> in debugfs.
>>>>
>>>
>>> These error messages are important for debugging.
>>> I suggest to keep in dev_err.
>>
>> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
>> end user not so much, since end users won't know the meaning of those values.
>>
>> For end users a "dma failure" message would be enough, then advanced users can enable
>> debug messages to see more.
> 
> OK.
> Got your point.
> 
>>>
>>> Moreover, could you give more information about debug counter?
>>> I don't get your point.
>>> Do you suggest to accumulate the total count of DMA errors?
>>
>>
>> Yes, you could have a debugfs entry with error counters like:
>>
>> cat /debugfs/mtk_isp/dma_err
>> 8
>>
>> So it is easier if this error happens very frequent or not.
>> In the rkisp1 case we added:
>>
>> /debugfs/rkisp1/data_loss
>> /debugfs/rkisp1/pic_size_error
>> /debugfs/rkisp1/mipi_error
>> /debugfs/rkisp1/stats_error
>> /debugfs/rkisp1/mp_stop_timeout
>> /debugfs/rkisp1/sp_stop_timeout
>> /debugfs/rkisp1/mp_frame_drop
>> /debugfs/rkisp1/sp_frame_drop
>>
>> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
>> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
>>
>> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
>>
> 
> Ok, we will follow your suggestion.
> 
> 
> [snip]
> 
>>>>> +return;
>>>>> +
>>>>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
>>>>> +
>>>>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
>>>>> +struct vb2_buffer *vb;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct mtk_cam_video_device *node;
>>>>> +
>>>>> +if (!vb2_request_object_is_buffer(obj))
>>>>> +continue;
>>>>> +vb = container_of(obj, struct vb2_buffer, req_obj);
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_del(&buf->list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
>>>>> +vb->timestamp = ts_eof;
>>>>> +else
>>>>> +vb->timestamp = req->timestamp;
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +return req;
>>>>> +}
>>>>> +}
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +
>>>>> +return NULL;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>>>>> +   unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>>>>> +list_del(&req->list);
>>>>> +break;
>>>>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space for frame drop */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>>>>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
>>>>> + req->frame_params.frame_seq_no);
>>>>
>>>> maybe a counter in debugfs instead of the warning.
>>>>
>>>
>>> Do you mean to add counter to accumulate the total count of drop frames?
>>
>> please see my comment above.
>>
> 
> Ok, add this in next patch.
> 
>>> Could we add this and also keep this warning message?
>>
>> Userspace would still continue to work when this happens, not sure if it is worthy
>> adding a warn, I would move it to dev_dbg() instead IMHO.
>>
> 
> Ok, revise in next patch.
> 
> [snip]
>>>>> +
>>>>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
>>>>> +     struct v4l2_pix_format_mplane *mp)
>>>>> +{
>>>>> +unsigned int bpl, ppl;
>>>>
>>>> bytes per line and pixels per line right?
>>>>
>>>
>>> Yes.
>>>
>>>>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>>>>
>>>> wouldn't be easier a get_pixel_bytes() function instead of bits?
>>>>
>>>
>>> Sorry. I didn't get the point.
>>> The unit of return value is bits, not bytes.
>>> Do you suggest move bpl & ppl calculation into get_pixel_bits() and
>>> rename to get_pixel_bytes()?
>>
>> Never mind, I misread it.
>>
> 
> Ok, we will skip this.
> 
> [snip]
>>>>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
>>>>> +int ret;
>>>>> +
>>>>> +/* Get sensor format configuration */
>>>>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>>>>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
>>>>> +if (ret) {
>>>>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +sd_width = sd_fmt.format.width;
>>>>> +sd_height = sd_fmt.format.height;
>>>>> +sd_code = sd_fmt.format.code;
>>>>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
>>>>> +sd_code);
>>>>
>>>> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
>>>> it should be configured from userspace.
>>>>
>>>
>>> Could you explain why?
>>> Moreover, how does configuration from user space?
>>
>> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
>>
>> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
>> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
>>
>> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
>> can be configured by userspace and part can't (which iirc should't be done in the media API).
>>
>> Do you need to use v4l2_device_register_subdev_nodes() ?
>>
>> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
>> https://patchwork.kernel.org/cover/11463183/
>>
>> which would be more appropriated if you don't want userspace to configure the whole pipeline.
>>
> 
> The purpose of P1 sub-device is to provide V4L2 event subscribe with
> V4L2_EVENT_FRAME_SYNC event for user space. It is the same
> implementation as blow link.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853
> 
> As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
> precisely.
> 
> [snip]
> 
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
>>>>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
>>>>> +
>>>>> +/* added the buffer into the tracking list */
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_add_tail(&buf->list, &node->buf_list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +
>>>>> +/* update buffer internal address */
>>>>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
>>>>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
>>>>
>>>> isn't it an issue if userspace queue two buffers for the same video device in the same request?
>>>>
>>>> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
>>>> will be at req->frame_params.dma_bufs[buf->node_id], no?
>>>>
>>>> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
>>>> buffer?
>>>>
>>>> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
>>>>
>>>
>>> 1. yes, it is a issue if userspace queues two buffers for the same video
>>> device with the same request FD.
>>>
>>> 2. All buffers which are belonged different to different video devices
>>> in the request list will be updated to req->frame_params.dma_bufs by
>>> buf->node_id.
>>>
>>> 3. It is not allowed for userspace to queue partial buffers for all
>>> enabled video devices. If it happens, it may trigger DMA errors for this
>>> request.
>>
>> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
>>
> 
> For case 1, it is handled in the vb2_queue_or_prepare_buf.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

Ok, thanks for the link.

> 
> For case 3, I need to correct my previous answer. This behavior should
> be OK for outputted DMA. We have frame buffer controller for each
> outputted DMAs. So it means the tuning buffer node is mandatory for each
> request, other nodes are optional. We will implement this
> in .req_validate to check.
> 
>>>
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +dma_addr_t addr;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +buf->node_id = node->id;
>>>>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
>>>>> +buf->scp_addr = 0;
>>>>> +
>>>>> +/* SCP address is only valid for meta input buffer */
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return 0;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +/* Use coherent address to get iova address */
>>>>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
>>>>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
>>>>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
>>>>> +return -EFAULT;
>>>>> +}
>>>>> +buf->scp_addr = buf->daddr;
>>>>> +buf->daddr = addr;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +if (vb2_plane_size(vb, 0) < size) {
>>>>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
>>>>> +vb2_plane_size(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
>>>>> +if (vb2_get_plane_payload(vb, 0) != size) {
>>>>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
>>>>> +vb2_get_plane_payload(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +v4l2_buf->field = V4L2_FIELD_NONE;
>>>>> +vb2_set_plane_payload(vb, 0, size);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +dma_unmap_page_attrs(dev, buf->daddr,
>>>>> +     vb->planes[0].length,
>>>>> +     DMA_BIDIRECTIONAL,
>>>>> +     DMA_ATTR_SKIP_CPU_SYNC);
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +
>>>>> +dev_dbg(cam->dev, "%s\n", __func__);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
>>>>> +   unsigned int *num_buffers,
>>>>> +   unsigned int *num_planes,
>>>>> +   unsigned int sizes[],
>>>>> +   struct device *alloc_devs[])
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +unsigned int max_buffer_count = node->desc.max_buf_count;
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +/* Check the limitation of buffer size */
>>>>> +if (max_buffer_count)
>>>>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
>>>>> +
>>>>> +if (node->desc.smem_alloc)
>>>>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>>>>> +
>>>>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
>>>>> +if (*num_planes) {
>>>>> +if (sizes[0] < size || *num_planes != 1)
>>>>> +return -EINVAL;
>>>>> +} else {
>>>>> +*num_planes = 1;
>>>>> +sizes[0] = size;
>>>>> +}
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
>>>>> +   struct mtk_cam_video_device *node,
>>>>> +   enum vb2_buffer_state state)
>>>>> +{
>>>>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
>>>>> +list_del(&buf->list);
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
>>>>> +       unsigned int count)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!node->enabled) {
>>>>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
>>>>> +ret = -ENOLINK;
>>>>> +goto fail_ret_buf;
>>>>> +}
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +/* Start streaming of the whole pipeline now*/
>>>>> +if (!cam->pipeline.streaming_count) {
>>>>
>>>> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
>>>>
>>>
>>> The check is designed to start the media pipeline when we start
>>> streaming on the first node. You could refer the detail in below link.
>>>
>>> https://patchwork.kernel.org/patch/10985819/
>>
>> right, ok, this is when enabling streaming from multiple nodes.
>>
>> media_pipeline_start() is usually called for every stream that starts.
>>
>> So cam->pipeline.streaming_count can reflect the number of streams enabled.
>>
>> So maybe you don't need cam->stream_count.
>>
> 
> Ok, revise in next patch.
> 
>>>
>>>
>>>>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
>>>>> +goto fail_unlock;
>>>>> +}
>>>>> +mtk_cam_dev_init_stream(cam);
>>>>> +ret = mtk_isp_hw_init(cam);
>>
>> Would it make sense to move this to s_stream in the ISP's subdevice ?
>>
> 
> No, we like to initialize our ISP firmware here when the first video
> node is streaming on. It is too late to initialize when all video nodes
> are streaming-on.
> 
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init HW:%d\n", ret);
>>>>> +goto fail_stop_pipeline;
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +/* Media links are fixed after media_pipeline_start */
>>>>> +cam->stream_count++;
>>>>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
>>>>> +cam->enabled_count);
>>>>> +if (cam->stream_count < cam->enabled_count) {
>>
>> I'm also wondering, since you need to wait for all the enabled video devices
>> to start streaming, shouldn't this be done inside a request? So you can enable
>> all of them at once?
>>
>> Also, like this you wouldn't need to check enabled links to query for enabled video
>> nodes, you can just enable the ones in the request.
>>
>> make sense?
>>
> 
> Sorry, I didn't get your point about this comment.
> Which request could we handle this logic?
> Do you mean move this logic into mtk_cam_req_queue function?

Sorry me, I thought we could use request api to call VIDIOC_STREAMON for multiple capture nodes,
but it seems we can't (please see my reply on the cover letter).

Regards,
Helen

> 
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +/* Stream on sub-devices node */
>>>>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
>>>>> +if (ret)
>>>>> +goto fail_no_stream;
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_no_stream:
>>>>> +cam->stream_count--;
>>>>> +fail_stop_pipeline:
>>>>> +if (cam->stream_count == 0)
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +fail_unlock:
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +fail_ret_buf:
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
>>>>> +cam->stream_count);
>>>>> +/* Check the first node to stream-off */
>>>>> +if (cam->stream_count == cam->enabled_count)
>>>>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
>>>>> +
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>>>>> +cam->stream_count--;
>>>>> +if (cam->stream_count) {
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return;
>>>>> +}
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +mtk_cam_dev_req_cleanup(cam);
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
>>>>> +   struct v4l2_capability *cap)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +
>>>>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
>>>>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
>>>>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
>>>>> + dev_name(cam->dev));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
>>>>> +   struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index >= node->desc.num_fmts)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt = node->vdev_fmt.fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
>>>>> +  struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +struct device *dev = cam->dev;
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +struct v4l2_format try_fmt;
>>>>> +
>>>>> +memset(&try_fmt, 0, sizeof(try_fmt));
>>>>> +try_fmt.type = f->type;
>>>>> +
>>>>> +/* Validate pixelformat */
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
>>>>> +if (!dev_fmt) {
>>>>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
>>>>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
>>>>> +}
>>>>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
>>>>> +
>>>>> +/* Validate image width & height range */
>>>>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
>>>>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
>>>>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
>>>>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
>>>>> +/* 4 bytes alignment for width */
>>>>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
>>>>> +
>>>>> +/* Only support one plane */
>>>>> +try_fmt.fmt.pix_mp.num_planes = 1;
>>>>> +
>>>>> +/* bytesperline & sizeimage calculation */
>>>>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
>>>>> +
>>>>> +/* Constant format fields */
>>>>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>>>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
>>>>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>>>> +
>>>>> +*f = try_fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (vb2_is_busy(node->vdev.queue)) {
>>>>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
>>>>> +return -EBUSY;
>>>>> +}
>>>>> +
>>>>> +/* Get the valid format */
>>>>> +mtk_cam_vidioc_try_fmt(file, fh, f);
>>>>> +/* Configure to video device */
>>>>> +node->vdev_fmt = *f;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
>>>>> +  struct v4l2_frmsizeenum *sizes)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
>>>>> +if (!dev_fmt || sizes->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +sizes->type = node->desc.frmsizes->type;
>>>>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
>>>>> +       sizeof(sizes->stepwise));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
>>>>> +     struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
>>>>> +.subscribe_event = mtk_cam_sd_subscribe_event,
>>>>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
>>>>> +.s_stream =  mtk_cam_sd_s_stream,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
>>>>> +.core = &mtk_cam_subdev_core_ops,
>>>>> +.video = &mtk_cam_subdev_video_ops,
>>>>> +};
>>>>
>>>> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
>>>> I wonder if pad ops shouldn't be implemented too (to be verified).
>>>>
>>>
>>> Ok, I will investigate this.
>>>
>>>>> +
>>>>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
>>>>> +.link_setup = mtk_cam_media_link_setup,
>>>>> +.link_validate = v4l2_subdev_link_validate,
>>>>> +};
>>>>> +
>>>>> +static const struct vb2_ops mtk_cam_vb2_ops = {
>>>>> +.queue_setup = mtk_cam_vb2_queue_setup,
>>>>> +.wait_prepare = vb2_ops_wait_prepare,
>>>>> +.wait_finish = vb2_ops_wait_finish,
>>>>> +.buf_init = mtk_cam_vb2_buf_init,
>>>>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
>>>>> +.start_streaming = mtk_cam_vb2_start_streaming,
>>>>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
>>>>> +.buf_queue = mtk_cam_vb2_buf_queue,
>>>>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
>>>>> +.buf_request_complete = mtk_cam_vb2_request_complete,
>>>>> +};> +
>>>>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
>>>>> +.unlocked_ioctl = video_ioctl2,
>>>>> +.open = v4l2_fh_open,
>>>>> +.release = vb2_fop_release,
>>>>> +.poll = vb2_fop_poll,
>>>>> +.mmap = vb2_fop_mmap,
>>>>> +#ifdef CONFIG_COMPAT
>>>>> +.compat_ioctl32 = v4l2_compat_ioctl32,
>>>>> +#endif
>>>>> +};
>>>>> +
>>>>> +static const struct media_device_ops mtk_cam_media_ops = {
>>>>> +.req_alloc = mtk_cam_req_alloc,
>>>>> +.req_free = mtk_cam_req_free,
>>>>> +.req_validate = vb2_request_validate,
>>>>> +.req_queue = mtk_cam_req_queue,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
>>>>> +  struct media_device *media_dev)
>>>>> +{
>>>>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
>>>>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +media_dev->dev = cam->dev;
>>>>> +strscpy(media_dev->model, dev_driver_string(dev),
>>>>> +sizeof(media_dev->model));
>>>>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
>>>>> + "platform:%s", dev_name(dev));
>>>>> +media_dev->hw_revision = 0;
>>>>> +media_device_init(media_dev);
>>>>> +media_dev->ops = &mtk_cam_media_ops;
>>>>> +
>>>>> +ret = media_device_register(media_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register media device:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +/* Initialize subdev pads */
>>>>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
>>>>> +sizeof(*cam->subdev_pads),
>>>>> +GFP_KERNEL);
>>>>> +if (!cam->subdev_pads) {
>>>>> +dev_err(dev, "failed to allocate subdev_pads\n");
>>>>> +ret = -ENOMEM;
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
>>>>> +     cam->subdev_pads);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
>>>>> +for (i = 0; i < num_pads; i++)
>>>>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Customize the last one pad as CIO sink pad. */
>>>>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int
>>>>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
>>>>> +      struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct video_device *vdev = &node->vdev;
>>>>> +struct vb2_queue *vbq = &node->vbq;
>>>>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
>>>>> +unsigned int link_flags = node->desc.link_flags;
>>>>> +int ret;
>>>>> +
>>>>> +/* Initialize mtk_cam_video_device */
>>>>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
>>>>> +node->enabled = true;
>>>>> +else
>>>>> +node->enabled = false;
>>>>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
>>>>> +
>>>>> +cam->subdev_pads[node->id].flags = output ?
>>>>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Initialize media entities */
>>>>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +/* Initialize vbq */
>>>>> +vbq->type = node->desc.buf_type;
>>>>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
>>>>> +vbq->io_modes = VB2_MMAP;
>>>>> +else
>>>>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
>>>>> +
>>>>> +if (node->desc.smem_alloc) {
>>>>> +vbq->bidirectional = 1;
>>>>> +vbq->dev = cam->smem_dev;
>>>>> +} else {
>>>>> +vbq->dev = dev;
>>>>> +}
>>>>> +vbq->ops = &mtk_cam_vb2_ops;
>>>>> +vbq->mem_ops = &vb2_dma_contig_memops;
>>>>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
>>>>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
>>>>> +if (output)
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
>>>>> +else
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>>>>> +/* No minimum buffers limitation */
>>>>> +vbq->min_buffers_needed = 0;
>>>>> +vbq->drv_priv = cam;
>>>>> +vbq->lock = &node->vdev_lock;
>>>>> +vbq->supports_requests = true;
>>>>> +vbq->requires_requests = true;
>>>>> +
>>>>> +ret = vb2_queue_init(vbq);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
>>>>> +goto fail_media_clean;
>>>>> +}
>>>>> +
>>>>> +/* Initialize vdev */
>>>>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
>>>>> + dev_driver_string(dev), node->desc.name);
>>>>> +/* set cap/type/ioctl_ops of the video device */
>>>>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
>>>>> +vdev->ioctl_ops = node->desc.ioctl_ops;
>>>>> +vdev->fops = &mtk_cam_v4l2_fops;
>>>>> +vdev->release = video_device_release_empty;
>>>>> +vdev->lock = &node->vdev_lock;
>>>>> +vdev->v4l2_dev = &cam->v4l2_dev;
>>>>> +vdev->queue = &node->vbq;
>>>>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
>>>>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
>>>>> +vdev->entity.ops = NULL;
>>>>> +video_set_drvdata(vdev, cam);
>>>>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
>>>>> +
>>>>> +/* Initialize miscellaneous variables */
>>>>> +mutex_init(&node->vdev_lock);
>>>>> +INIT_LIST_HEAD(&node->buf_list);
>>>>> +spin_lock_init(&node->buf_list_lock);
>>>>> +
>>>>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register vde:%d\n", ret);
>>>>> +goto fail_vb2_rel;
>>>>> +}
>>>>> +
>>>>> +/* Create link between video node and the subdev pad */
>>>>> +if (output) {
>>>>> +ret = media_create_pad_link(&vdev->entity, 0,
>>>>> +    &cam->subdev.entity,
>>>>> +    node->id, link_flags);
>>>>> +} else {
>>>>> +ret = media_create_pad_link(&cam->subdev.entity,
>>>>> +    node->id, &vdev->entity, 0,
>>>>> +    link_flags);
>>>>> +}
>>>>
>>>> No need for the curly braces.
>>>>
>>>
>>> Revised in next patch.
>>>
>>>>> +if (ret)
>>>>> +goto fail_vdev_ureg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_ureg:
>>>>> +video_unregister_device(vdev);
>>>>> +fail_vb2_rel:
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +vb2_queue_release(vbq);
>>>>> +fail_media_clean:
>>>>> +media_entity_cleanup(&vdev->entity);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void
>>>>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +video_unregister_device(&node->vdev);
>>>>> +vb2_queue_release(&node->vbq);
>>>>> +media_entity_cleanup(&node->vdev.entity);
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +/* Set up media device & pads */
>>>>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
>>>>> +
>>>>> +/* Set up v4l2 device */
>>>>> +cam->v4l2_dev.mdev = &cam->media_dev;
>>>>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
>>>>> +
>>>>> +/* Initialize subdev */
>>>>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
>>>>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>>>>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
>>>>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>>>>> +V4L2_SUBDEV_FL_HAS_EVENTS;
>>>>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
>>>>> + "%s", dev_driver_string(dev));
>>>>> +v4l2_set_subdevdata(&cam->subdev, cam);
>>>>> +
>>>>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
>>>>> +goto fail_clean_media_entiy;
>>>>> +}
>>>>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
>>>>> +
>>>>> +/* Create video nodes and links */
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>>>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
>>>>> +
>>>>> +node->id = node->desc.id;
>>>>> +ret = mtk_cam_video_register_device(cam, node);
>>>>> +if (ret)
>>>>> +goto fail_vdev_unreg;
>>>>> +}
>>>>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_unreg:
>>>>> +for (i--; i >= 0; i--)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +fail_clean_media_entiy:
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int i;
>>>>> +
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +
>>>>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
>>>>> +v4l2_device_unregister_subdev(&cam->subdev);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
>>>>> +      struct v4l2_subdev *sd,
>>>>> +      struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>>>>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +cam->seninf = sd;
>>>>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
>>>>> +struct v4l2_subdev *sd,
>>>>> +struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +cam->seninf = NULL;
>>>>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!cam->seninf) {
>>>>> +dev_err(dev, "No seninf subdev\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
>>>>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
>>>>> +    MEDIA_LNK_FL_IMMUTABLE |
>>>>> +    MEDIA_LNK_FL_ENABLED);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
>>>>> +cam->seninf->entity.name, cam->subdev.entity.name,
>>>>> +ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
>>>>> +.bound = mtk_cam_dev_notifier_bound,
>>>>> +.unbind = mtk_cam_dev_notifier_unbind,
>>>>> +.complete = mtk_cam_dev_notifier_complete,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +v4l2_async_notifier_init(&cam->notifier);
>>>>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
>>>>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
>>>>
>>>> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
>>>>
>>>> Regards,
>>>> Helen
>>>>
>>>
>>> Ok, we will investigate how to do.
>>>
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
>>>>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
>>>>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +v4l2_async_notifier_unregister(&cam->notifier);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
>>>>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
>>>>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
>>>>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
>>>>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>>>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};> +
>>>>> +static const struct v4l2_format meta_fmts[] = {
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
>>>>> +.buffersize = 512 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_3A,
>>>>> +.buffersize = 1200 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_AF,
>>>>> +.buffersize = 640 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
>>>>> +.buffersize = 288 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
>>>>> +.buffersize = 256,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format stream_out_fmts[] = {
>>>>> +/* This is a default image format */
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format bin_out_fmts[] = {
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc output_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_IN_0,
>>>>> +.name = "meta input",
>>>>> +.cap = V4L2_CAP_META_OUTPUT,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = true,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 0,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc capture_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +.name = "main stream",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_IMGO,
>>>>> +.fmts = stream_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +.name = "packed out",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = 0,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_RRZO,
>>>>> +.fmts = bin_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_0,
>>>>> +.name = "partial meta 0",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AAO | R_FLKO | R_PSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 1,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_1,
>>>>> +.name = "partial meta 1",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AFO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 2,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_2,
>>>>> +.name = "partial meta 2",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LCSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 3,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_3,
>>>>> +.name = "partial meta 3",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LMVO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 4,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +/* The helper to configure the device context */
>>>>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +unsigned int node_idx;
>>>>> +int i;
>>>>> +
>>>>> +node_idx = 0;
>>>>> +/* Setup the output queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
>>>>> +
>>>>> +/* Setup the capture queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
>>>>> +}
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int ret;
>>>>> +
>>>>> +cam->dev = &pdev->dev;
>>>>> +mtk_cam_dev_queue_setup(cam);
>>>>> +
>>>>> +spin_lock_init(&cam->pending_job_lock);
>>>>> +spin_lock_init(&cam->running_job_lock);
>>>>> +INIT_LIST_HEAD(&cam->pending_job_list);
>>>>> +INIT_LIST_HEAD(&cam->running_job_list);
>>>>> +mutex_init(&cam->op_lock);
>>>>> +
>>>>> +/* v4l2 sub-device registration */
>>>>> +ret = mtk_cam_v4l2_register(cam);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +
>>>>> +ret = mtk_cam_v4l2_async_register(cam);
>>>>> +if (ret)
>>>>> +goto fail_v4l2_unreg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_v4l2_unreg:
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +mtk_cam_v4l2_async_unregister(cam);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +}
>>>>> +
>>>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> new file mode 100644
>>>>> index 000000000000..0a340a1e65ea
>>>>> --- /dev/null
>>>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> @@ -0,0 +1,244 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>>> +/*
>>>>> + * Copyright (c) 2019 MediaTek Inc.
>>>>> + */
>>>>> +
>>>>> +#ifndef __MTK_CAM_H__
>>>>> +#define __MTK_CAM_H__
>>>>> +
>>>>> +#include <linux/device.h>
>>>>> +#include <linux/types.h>
>>>>> +#include <linux/platform_device.h>
>>>>> +#include <linux/spinlock.h>
>>>>> +#include <linux/videodev2.h>
>>>>> +#include <media/v4l2-device.h>
>>>>> +#include <media/v4l2-ctrls.h>
>>>>> +#include <media/v4l2-subdev.h>
>>>>> +#include <media/videobuf2-core.h>
>>>>> +#include <media/videobuf2-v4l2.h>
>>>>> +
>>>>> +#include "mtk_cam-ipi.h"
>>>>> +
>>>>> +#define IMG_MAX_WIDTH5376
>>>>> +#define IMG_MAX_HEIGHT4032
>>>>> +#define IMG_MIN_WIDTH80
>>>>> +#define IMG_MIN_HEIGHT60
>>>>> +
>>>>> +/*
>>>>> + * ID enum value for struct mtk_cam_dev_node_desc:id
>>>>> + * or mtk_cam_video_device:id
>>>>> + */
>>>>> +enum  {
>>>>> +MTK_CAM_P1_META_IN_0 = 0,
>>>>> +MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +MTK_CAM_P1_META_OUT_0,
>>>>> +MTK_CAM_P1_META_OUT_1,
>>>>> +MTK_CAM_P1_META_OUT_2,
>>>>> +MTK_CAM_P1_META_OUT_3,
>>>>> +MTK_CAM_P1_TOTAL_NODES
>>>>> +};
>>>>> +
>>>>> +/* Supported image format list */
>>>>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
>>>>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
>>>>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
>>>>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
>>>>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
>>>>> +
>>>>> +/* Supported bayer pixel order */
>>>>> +#define MTK_CAM_RAW_PXL_ID_B0
>>>>> +#define MTK_CAM_RAW_PXL_ID_GB1
>>>>> +#define MTK_CAM_RAW_PXL_ID_GR2
>>>>> +#define MTK_CAM_RAW_PXL_ID_R3
>>>>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
>>>>> + *
>>>>> + * @frame_seq_no: The frame sequence of frame in driver layer.
>>>>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_p1_frame_param {
>>>>> +unsigned int frame_seq_no;
>>>>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_request - MTK camera device request.
>>>>> + *
>>>>> + * @req: Embedded struct media request.
>>>>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
>>>>> + * @frame_work: work queue entry for frame transmission to SCP.
>>>>> + * @list: List entry of the object for @struct mtk_cam_dev:
>>>>> + *        pending_job_list or running_job_list.
>>>>> + * @timestamp: Start of frame timestamp in ns
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_request {
>>>>> +struct media_request req;
>>>>> +struct mtk_p1_frame_param frame_params;
>>>>> +struct work_struct frame_work;
>>>>> +struct list_head list;
>>>>> +u64 timestamp;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
>>>>> + *
>>>>> + * @vbb: Embedded struct vb2_v4l2_buffer.
>>>>> + * @list: List entry of the object for @struct mtk_cam_video_device:
>>>>> + *        buf_list.
>>>>> + * @daddr: The DMA address of this buffer.
>>>>> + * @scp_addr: The SCP address of this buffer which
>>>>> + *            is only supported for meta input node.
>>>>> + * @node_id: The vidoe node id which this buffer belongs to.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_buffer {
>>>>> +struct vb2_v4l2_buffer vbb;
>>>>> +struct list_head list;
>>>>> +/* Intenal part */
>>>>> +dma_addr_t daddr;
>>>>> +dma_addr_t scp_addr;
>>>>> +unsigned int node_id;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
>>>>> + *
>>>>> + * @id: id of the node
>>>>> + * @name: name of the node
>>>>> + * @cap: supported V4L2 capabilities
>>>>> + * @buf_type: supported V4L2 buffer type
>>>>> + * @dma_port: the dma ports associated to the node
>>>>> + * @link_flags: default media link flags
>>>>> + * @smem_alloc: using the smem_dev as alloc device or not
>>>>> + * @image: true for image node, false for meta node
>>>>> + * @num_fmts: the number of supported node formats
>>>>> + * @default_fmt_idx: default format of this node
>>>>> + * @max_buf_count: maximum VB2 buffer count
>>>>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
>>>>> + * @fmts: supported format
>>>>> + * @frmsizes: supported V4L2 frame size number
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_node_desc {
>>>>> +u8 id;
>>>>> +const char *name;
>>>>> +u32 cap;
>>>>> +u32 buf_type;
>>>>> +u32 dma_port;
>>>>> +u32 link_flags;
>>>>> +u8 smem_alloc:1;
>>>>> +u8 image:1;
>>>>> +u8 num_fmts;
>>>>> +u8 default_fmt_idx;
>>>>> +u8 max_buf_count;
>>>>> +const struct v4l2_ioctl_ops *ioctl_ops;
>>>>> +const struct v4l2_format *fmts;
>>>>> +const struct v4l2_frmsizeenum *frmsizes;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_video_device - Mediatek video device structure
>>>>> + *
>>>>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
>>>>> + * @enabled: Indicate the video device is enabled or not
>>>>> + * @desc: The node description of video device
>>>>> + * @vdev_fmt: The V4L2 format of video device
>>>>> + * @vdev_pad: The media pad graph object of video device
>>>>> + * @vbq: A videobuf queue of video device
>>>>> + * @vdev: The video device instance
>>>>> + * @vdev_lock: Serializes vb2 queue and video device operations
>>>>> + * @buf_list: List for enqueue buffers
>>>>> + * @buf_list_lock: Lock used to protect buffer list.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_video_device {
>>>>> +unsigned int id;
>>>>> +unsigned int enabled;
>>>>> +struct mtk_cam_dev_node_desc desc;
>>>>> +struct v4l2_format vdev_fmt;
>>>>> +struct media_pad vdev_pad;
>>>>> +struct vb2_queue vbq;
>>>>> +struct video_device vdev;
>>>>> +/* Serializes vb2 queue and video device operations */
>>>>> +struct mutex vdev_lock;
>>>>> +struct list_head buf_list;
>>>>> +/* Lock used to protect buffer list */
>>>>> +spinlock_t buf_list_lock;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev - Mediatek camera device structure.
>>>>> + *
>>>>> + * @dev: Pointer to device.
>>>>> + * @smem_pdev: Pointer to shared memory device.
>>>>> + * @pipeline: Media pipeline information.
>>>>> + * @media_dev: Media device instance.
>>>>> + * @subdev: The V4L2 sub-device instance.
>>>>> + * @v4l2_dev: The V4L2 device driver instance.
>>>>> + * @notifier: The v4l2_device notifier data.
>>>>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
>>>>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
>>>>> + * @seninf: Pointer to the seninf sub-device.
>>>>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
>>>>> + * @streaming: Indicate the overall streaming status is on or off.
>>>>> + * @enabled_dmas: The enabled dma port information when streaming on.
>>>>> + * @enabled_count: Number of enabled video nodes
>>>>> + * @stream_count: Number of streaming video nodes
>>>>> + * @running_job_count: Nunber of running jobs in the HW driver.
>>>>> + * @pending_job_list: List to keep the media requests before en-queue into
>>>>> + *                    HW driver.
>>>>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
>>>>> + * @running_job_list: List to keep the media requests after en-queue into
>>>>> + *                    HW driver.
>>>>> + * @running_job_lock: Protect the running_job_list data.
>>>>> + * @op_lock: Serializes driver's VB2 callback operations.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev {
>>>>> +struct device *dev;
>>>>> +struct device *smem_dev;
>>>>> +struct media_pipeline pipeline;
>>>>> +struct media_device media_dev;
>>>>> +struct v4l2_subdev subdev;
>>>>> +struct v4l2_device v4l2_dev;
>>>>> +struct v4l2_async_notifier notifier;
>>>>> +struct media_pad *subdev_pads;
>>>>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
>>>>> +struct v4l2_subdev *seninf;
>>>>> +struct v4l2_subdev *sensor;
>>>>> +unsigned int streaming;
>>>>> +unsigned int enabled_dmas;
>>>>> +unsigned int enabled_count;
>>>>> +unsigned int stream_count;
>>>>> +unsigned int running_job_count;
>>>>> +struct list_head pending_job_list;
>>>>> +/* Protect the pending_job_list data */
>>>>> +spinlock_t pending_job_lock;
>>>>> +struct list_head running_job_list;
>>>>> +/* Protect the running_job_list data & running_job_count */
>>>>> +spinlock_t running_job_lock;
>>>>> +/* Serializes driver's VB2 callback operations */
>>>>> +struct mutex op_lock;
>>>>> +};
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
>>>>> +   unsigned int frame_seq_no);
>>>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
>>>>> +  unsigned int frame_seq_no);
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no);
>>>>> +
>>>>> +#endif /* __MTK_CAM_H__ */
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
>>>
>>
>> Regards,
>> Helen
> 
> Thanks for your comment.
> 
> Best regards,
> 
> 
> Jungo
> 

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-05-05 15:38                 ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:38 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport



On 5/4/20 9:27 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>> On 4/8/20 11:05 PM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comments.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hello Jungo,
>>>>
>>>> I was taking a look at this patch (thanks for the work),
>>>> I didn't look in deep details, but I have some comments, please see
>>>> below. I hope it helps.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> This patch adds the Mediatek ISP P1 HW control device driver.
>>>>> It handles the ISP HW configuration, provides interrupt handling and
>>>>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
>>>>> implement standard V4L2 video driver that utilizes V4L2 and media
>>>>> framework APIs. It supports one media device, one sub-device and
>>>>> several video devices during initialization. Moreover, it also connects
>>>>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
>>>>> co-process via SCP communication to compose ISP registers in the
>>>>> firmware.
>>>>>
>>>>> (The current metadata interface used in meta input and partial
>>>>> meta nodes is only a temporary solution to kick off the driver
>>>>> development and is not ready to be reviewed yet.)
>>>>>
>>>>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
>>>>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
>>>>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
>>>>> ---
>>>>> Changes from v6:
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>>>>>  - Correct auto suspend timer value for suspend/resume issue
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>>>>>  - Fix KE due to no sen-inf sub-device
>>>>> ---
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>
>>>> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
>>>> capture, which for metadata, and what is being used to deal with requests.
>>>>
>>>> It would make it easier to review imho.
>>>>
>>>
>>> For file structure design, it was reviewed in the previous patch
>>> serials.
>>> e.g.
>>> https://patchwork.kernel.org/patch/10938137/
>>> If you think it is better, I will modify it.
>>
>> Right, I saw a suggestion to merge two files there.
>>
>> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
>> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
>> So it reflects a bit the topology.
>> But it is also up to you to see if it improves organization or not, it is just a suggestion.
>>
> 
> Ok, got your point.
> We will discuss how to do internally.
> 
> [snip]
>>>>> +isp_composer_hw_init(p1_dev);
>>>>> +
>>>>> +p1_dev->enqueued_frame_seq_no = 0;
>>>>> +p1_dev->dequeued_frame_seq_no = 0;
>>>>> +p1_dev->composed_frame_seq_no = 0;
>>>>> +p1_dev->sof_count = 0;
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>>>> +
>>>>> +isp_composer_hw_deinit(p1_dev);
>>>>> +pm_runtime_mark_last_busy(dev);
>>>>> +pm_runtime_put_autosuspend(dev);
>>>>> +rproc_shutdown(p1_dev->rproc_handle);
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>>>>> + struct mtk_cam_dev_request *req)
>>>>> +{
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>>>> +
>>>>> +/* Accumulated frame sequence number */
>>>>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
>>>>> +
>>>>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>>>>> +queue_work(p1_dev->composer_wq, &req->frame_work);
>>>>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no,
>>>>> +cam->running_job_count);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>>>>> +       unsigned int dequeued_frame_seq_no)
>>>>> +{
>>>>> +dma_addr_t base_addr = p1_dev->composer_iova;
>>>>> +struct device *dev = p1_dev->dev;
>>>>> +struct mtk_cam_dev_request *req;
>>>>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
>>>>> +unsigned int addr_offset;
>>>>> +
>>>>> +/* Send V4L2_EVENT_FRAME_SYNC event */
>>>>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +
>>>>> +p1_dev->sof_count += 1;
>>>>> +/* Save frame information */
>>>>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
>>>>> +
>>>>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +if (req)
>>>>> +req->timestamp = ktime_get_boottime_ns();
>>>>> +
>>>>> +/* Update CQ base address if needed */
>>>>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no);
>>>>> +return;
>>>>> +}
>>>>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>>>>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>>>>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
>>>>> +{
>>>>> +u32 val;
>>>>> +
>>>>> +dev_err(p1_dev->dev,
>>>>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
>>>>> +dev_err(p1_dev->dev,
>>>>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
>>>>
>>>> I think if would be better to transfor those into dev_dbg and add a counter
>>>> in debugfs.
>>>>
>>>
>>> These error messages are important for debugging.
>>> I suggest to keep in dev_err.
>>
>> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
>> end user not so much, since end users won't know the meaning of those values.
>>
>> For end users a "dma failure" message would be enough, then advanced users can enable
>> debug messages to see more.
> 
> OK.
> Got your point.
> 
>>>
>>> Moreover, could you give more information about debug counter?
>>> I don't get your point.
>>> Do you suggest to accumulate the total count of DMA errors?
>>
>>
>> Yes, you could have a debugfs entry with error counters like:
>>
>> cat /debugfs/mtk_isp/dma_err
>> 8
>>
>> So it is easier if this error happens very frequent or not.
>> In the rkisp1 case we added:
>>
>> /debugfs/rkisp1/data_loss
>> /debugfs/rkisp1/pic_size_error
>> /debugfs/rkisp1/mipi_error
>> /debugfs/rkisp1/stats_error
>> /debugfs/rkisp1/mp_stop_timeout
>> /debugfs/rkisp1/sp_stop_timeout
>> /debugfs/rkisp1/mp_frame_drop
>> /debugfs/rkisp1/sp_frame_drop
>>
>> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
>> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
>>
>> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
>>
> 
> Ok, we will follow your suggestion.
> 
> 
> [snip]
> 
>>>>> +return;
>>>>> +
>>>>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
>>>>> +
>>>>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
>>>>> +struct vb2_buffer *vb;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct mtk_cam_video_device *node;
>>>>> +
>>>>> +if (!vb2_request_object_is_buffer(obj))
>>>>> +continue;
>>>>> +vb = container_of(obj, struct vb2_buffer, req_obj);
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_del(&buf->list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
>>>>> +vb->timestamp = ts_eof;
>>>>> +else
>>>>> +vb->timestamp = req->timestamp;
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +return req;
>>>>> +}
>>>>> +}
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +
>>>>> +return NULL;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>>>>> +   unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>>>>> +list_del(&req->list);
>>>>> +break;
>>>>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space for frame drop */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>>>>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
>>>>> + req->frame_params.frame_seq_no);
>>>>
>>>> maybe a counter in debugfs instead of the warning.
>>>>
>>>
>>> Do you mean to add counter to accumulate the total count of drop frames?
>>
>> please see my comment above.
>>
> 
> Ok, add this in next patch.
> 
>>> Could we add this and also keep this warning message?
>>
>> Userspace would still continue to work when this happens, not sure if it is worthy
>> adding a warn, I would move it to dev_dbg() instead IMHO.
>>
> 
> Ok, revise in next patch.
> 
> [snip]
>>>>> +
>>>>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
>>>>> +     struct v4l2_pix_format_mplane *mp)
>>>>> +{
>>>>> +unsigned int bpl, ppl;
>>>>
>>>> bytes per line and pixels per line right?
>>>>
>>>
>>> Yes.
>>>
>>>>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>>>>
>>>> wouldn't be easier a get_pixel_bytes() function instead of bits?
>>>>
>>>
>>> Sorry. I didn't get the point.
>>> The unit of return value is bits, not bytes.
>>> Do you suggest move bpl & ppl calculation into get_pixel_bits() and
>>> rename to get_pixel_bytes()?
>>
>> Never mind, I misread it.
>>
> 
> Ok, we will skip this.
> 
> [snip]
>>>>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
>>>>> +int ret;
>>>>> +
>>>>> +/* Get sensor format configuration */
>>>>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>>>>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
>>>>> +if (ret) {
>>>>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +sd_width = sd_fmt.format.width;
>>>>> +sd_height = sd_fmt.format.height;
>>>>> +sd_code = sd_fmt.format.code;
>>>>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
>>>>> +sd_code);
>>>>
>>>> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
>>>> it should be configured from userspace.
>>>>
>>>
>>> Could you explain why?
>>> Moreover, how does configuration from user space?
>>
>> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
>>
>> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
>> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
>>
>> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
>> can be configured by userspace and part can't (which iirc should't be done in the media API).
>>
>> Do you need to use v4l2_device_register_subdev_nodes() ?
>>
>> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
>> https://patchwork.kernel.org/cover/11463183/
>>
>> which would be more appropriated if you don't want userspace to configure the whole pipeline.
>>
> 
> The purpose of P1 sub-device is to provide V4L2 event subscribe with
> V4L2_EVENT_FRAME_SYNC event for user space. It is the same
> implementation as blow link.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853
> 
> As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
> precisely.
> 
> [snip]
> 
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
>>>>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
>>>>> +
>>>>> +/* added the buffer into the tracking list */
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_add_tail(&buf->list, &node->buf_list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +
>>>>> +/* update buffer internal address */
>>>>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
>>>>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
>>>>
>>>> isn't it an issue if userspace queue two buffers for the same video device in the same request?
>>>>
>>>> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
>>>> will be at req->frame_params.dma_bufs[buf->node_id], no?
>>>>
>>>> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
>>>> buffer?
>>>>
>>>> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
>>>>
>>>
>>> 1. yes, it is a issue if userspace queues two buffers for the same video
>>> device with the same request FD.
>>>
>>> 2. All buffers which are belonged different to different video devices
>>> in the request list will be updated to req->frame_params.dma_bufs by
>>> buf->node_id.
>>>
>>> 3. It is not allowed for userspace to queue partial buffers for all
>>> enabled video devices. If it happens, it may trigger DMA errors for this
>>> request.
>>
>> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
>>
> 
> For case 1, it is handled in the vb2_queue_or_prepare_buf.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

Ok, thanks for the link.

> 
> For case 3, I need to correct my previous answer. This behavior should
> be OK for outputted DMA. We have frame buffer controller for each
> outputted DMAs. So it means the tuning buffer node is mandatory for each
> request, other nodes are optional. We will implement this
> in .req_validate to check.
> 
>>>
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +dma_addr_t addr;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +buf->node_id = node->id;
>>>>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
>>>>> +buf->scp_addr = 0;
>>>>> +
>>>>> +/* SCP address is only valid for meta input buffer */
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return 0;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +/* Use coherent address to get iova address */
>>>>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
>>>>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
>>>>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
>>>>> +return -EFAULT;
>>>>> +}
>>>>> +buf->scp_addr = buf->daddr;
>>>>> +buf->daddr = addr;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +if (vb2_plane_size(vb, 0) < size) {
>>>>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
>>>>> +vb2_plane_size(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
>>>>> +if (vb2_get_plane_payload(vb, 0) != size) {
>>>>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
>>>>> +vb2_get_plane_payload(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +v4l2_buf->field = V4L2_FIELD_NONE;
>>>>> +vb2_set_plane_payload(vb, 0, size);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +dma_unmap_page_attrs(dev, buf->daddr,
>>>>> +     vb->planes[0].length,
>>>>> +     DMA_BIDIRECTIONAL,
>>>>> +     DMA_ATTR_SKIP_CPU_SYNC);
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +
>>>>> +dev_dbg(cam->dev, "%s\n", __func__);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
>>>>> +   unsigned int *num_buffers,
>>>>> +   unsigned int *num_planes,
>>>>> +   unsigned int sizes[],
>>>>> +   struct device *alloc_devs[])
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +unsigned int max_buffer_count = node->desc.max_buf_count;
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +/* Check the limitation of buffer size */
>>>>> +if (max_buffer_count)
>>>>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
>>>>> +
>>>>> +if (node->desc.smem_alloc)
>>>>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>>>>> +
>>>>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
>>>>> +if (*num_planes) {
>>>>> +if (sizes[0] < size || *num_planes != 1)
>>>>> +return -EINVAL;
>>>>> +} else {
>>>>> +*num_planes = 1;
>>>>> +sizes[0] = size;
>>>>> +}
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
>>>>> +   struct mtk_cam_video_device *node,
>>>>> +   enum vb2_buffer_state state)
>>>>> +{
>>>>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
>>>>> +list_del(&buf->list);
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
>>>>> +       unsigned int count)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!node->enabled) {
>>>>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
>>>>> +ret = -ENOLINK;
>>>>> +goto fail_ret_buf;
>>>>> +}
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +/* Start streaming of the whole pipeline now*/
>>>>> +if (!cam->pipeline.streaming_count) {
>>>>
>>>> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
>>>>
>>>
>>> The check is designed to start the media pipeline when we start
>>> streaming on the first node. You could refer the detail in below link.
>>>
>>> https://patchwork.kernel.org/patch/10985819/
>>
>> right, ok, this is when enabling streaming from multiple nodes.
>>
>> media_pipeline_start() is usually called for every stream that starts.
>>
>> So cam->pipeline.streaming_count can reflect the number of streams enabled.
>>
>> So maybe you don't need cam->stream_count.
>>
> 
> Ok, revise in next patch.
> 
>>>
>>>
>>>>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
>>>>> +goto fail_unlock;
>>>>> +}
>>>>> +mtk_cam_dev_init_stream(cam);
>>>>> +ret = mtk_isp_hw_init(cam);
>>
>> Would it make sense to move this to s_stream in the ISP's subdevice ?
>>
> 
> No, we like to initialize our ISP firmware here when the first video
> node is streaming on. It is too late to initialize when all video nodes
> are streaming-on.
> 
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init HW:%d\n", ret);
>>>>> +goto fail_stop_pipeline;
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +/* Media links are fixed after media_pipeline_start */
>>>>> +cam->stream_count++;
>>>>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
>>>>> +cam->enabled_count);
>>>>> +if (cam->stream_count < cam->enabled_count) {
>>
>> I'm also wondering, since you need to wait for all the enabled video devices
>> to start streaming, shouldn't this be done inside a request? So you can enable
>> all of them at once?
>>
>> Also, like this you wouldn't need to check enabled links to query for enabled video
>> nodes, you can just enable the ones in the request.
>>
>> make sense?
>>
> 
> Sorry, I didn't get your point about this comment.
> Which request could we handle this logic?
> Do you mean move this logic into mtk_cam_req_queue function?

Sorry me, I thought we could use request api to call VIDIOC_STREAMON for multiple capture nodes,
but it seems we can't (please see my reply on the cover letter).

Regards,
Helen

> 
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +/* Stream on sub-devices node */
>>>>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
>>>>> +if (ret)
>>>>> +goto fail_no_stream;
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_no_stream:
>>>>> +cam->stream_count--;
>>>>> +fail_stop_pipeline:
>>>>> +if (cam->stream_count == 0)
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +fail_unlock:
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +fail_ret_buf:
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
>>>>> +cam->stream_count);
>>>>> +/* Check the first node to stream-off */
>>>>> +if (cam->stream_count == cam->enabled_count)
>>>>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
>>>>> +
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>>>>> +cam->stream_count--;
>>>>> +if (cam->stream_count) {
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return;
>>>>> +}
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +mtk_cam_dev_req_cleanup(cam);
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
>>>>> +   struct v4l2_capability *cap)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +
>>>>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
>>>>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
>>>>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
>>>>> + dev_name(cam->dev));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
>>>>> +   struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index >= node->desc.num_fmts)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt = node->vdev_fmt.fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
>>>>> +  struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +struct device *dev = cam->dev;
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +struct v4l2_format try_fmt;
>>>>> +
>>>>> +memset(&try_fmt, 0, sizeof(try_fmt));
>>>>> +try_fmt.type = f->type;
>>>>> +
>>>>> +/* Validate pixelformat */
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
>>>>> +if (!dev_fmt) {
>>>>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
>>>>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
>>>>> +}
>>>>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
>>>>> +
>>>>> +/* Validate image width & height range */
>>>>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
>>>>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
>>>>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
>>>>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
>>>>> +/* 4 bytes alignment for width */
>>>>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
>>>>> +
>>>>> +/* Only support one plane */
>>>>> +try_fmt.fmt.pix_mp.num_planes = 1;
>>>>> +
>>>>> +/* bytesperline & sizeimage calculation */
>>>>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
>>>>> +
>>>>> +/* Constant format fields */
>>>>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>>>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
>>>>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>>>> +
>>>>> +*f = try_fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (vb2_is_busy(node->vdev.queue)) {
>>>>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
>>>>> +return -EBUSY;
>>>>> +}
>>>>> +
>>>>> +/* Get the valid format */
>>>>> +mtk_cam_vidioc_try_fmt(file, fh, f);
>>>>> +/* Configure to video device */
>>>>> +node->vdev_fmt = *f;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
>>>>> +  struct v4l2_frmsizeenum *sizes)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
>>>>> +if (!dev_fmt || sizes->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +sizes->type = node->desc.frmsizes->type;
>>>>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
>>>>> +       sizeof(sizes->stepwise));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
>>>>> +     struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
>>>>> +.subscribe_event = mtk_cam_sd_subscribe_event,
>>>>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
>>>>> +.s_stream =  mtk_cam_sd_s_stream,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
>>>>> +.core = &mtk_cam_subdev_core_ops,
>>>>> +.video = &mtk_cam_subdev_video_ops,
>>>>> +};
>>>>
>>>> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
>>>> I wonder if pad ops shouldn't be implemented too (to be verified).
>>>>
>>>
>>> Ok, I will investigate this.
>>>
>>>>> +
>>>>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
>>>>> +.link_setup = mtk_cam_media_link_setup,
>>>>> +.link_validate = v4l2_subdev_link_validate,
>>>>> +};
>>>>> +
>>>>> +static const struct vb2_ops mtk_cam_vb2_ops = {
>>>>> +.queue_setup = mtk_cam_vb2_queue_setup,
>>>>> +.wait_prepare = vb2_ops_wait_prepare,
>>>>> +.wait_finish = vb2_ops_wait_finish,
>>>>> +.buf_init = mtk_cam_vb2_buf_init,
>>>>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
>>>>> +.start_streaming = mtk_cam_vb2_start_streaming,
>>>>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
>>>>> +.buf_queue = mtk_cam_vb2_buf_queue,
>>>>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
>>>>> +.buf_request_complete = mtk_cam_vb2_request_complete,
>>>>> +};> +
>>>>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
>>>>> +.unlocked_ioctl = video_ioctl2,
>>>>> +.open = v4l2_fh_open,
>>>>> +.release = vb2_fop_release,
>>>>> +.poll = vb2_fop_poll,
>>>>> +.mmap = vb2_fop_mmap,
>>>>> +#ifdef CONFIG_COMPAT
>>>>> +.compat_ioctl32 = v4l2_compat_ioctl32,
>>>>> +#endif
>>>>> +};
>>>>> +
>>>>> +static const struct media_device_ops mtk_cam_media_ops = {
>>>>> +.req_alloc = mtk_cam_req_alloc,
>>>>> +.req_free = mtk_cam_req_free,
>>>>> +.req_validate = vb2_request_validate,
>>>>> +.req_queue = mtk_cam_req_queue,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
>>>>> +  struct media_device *media_dev)
>>>>> +{
>>>>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
>>>>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +media_dev->dev = cam->dev;
>>>>> +strscpy(media_dev->model, dev_driver_string(dev),
>>>>> +sizeof(media_dev->model));
>>>>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
>>>>> + "platform:%s", dev_name(dev));
>>>>> +media_dev->hw_revision = 0;
>>>>> +media_device_init(media_dev);
>>>>> +media_dev->ops = &mtk_cam_media_ops;
>>>>> +
>>>>> +ret = media_device_register(media_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register media device:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +/* Initialize subdev pads */
>>>>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
>>>>> +sizeof(*cam->subdev_pads),
>>>>> +GFP_KERNEL);
>>>>> +if (!cam->subdev_pads) {
>>>>> +dev_err(dev, "failed to allocate subdev_pads\n");
>>>>> +ret = -ENOMEM;
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
>>>>> +     cam->subdev_pads);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
>>>>> +for (i = 0; i < num_pads; i++)
>>>>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Customize the last one pad as CIO sink pad. */
>>>>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int
>>>>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
>>>>> +      struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct video_device *vdev = &node->vdev;
>>>>> +struct vb2_queue *vbq = &node->vbq;
>>>>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
>>>>> +unsigned int link_flags = node->desc.link_flags;
>>>>> +int ret;
>>>>> +
>>>>> +/* Initialize mtk_cam_video_device */
>>>>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
>>>>> +node->enabled = true;
>>>>> +else
>>>>> +node->enabled = false;
>>>>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
>>>>> +
>>>>> +cam->subdev_pads[node->id].flags = output ?
>>>>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Initialize media entities */
>>>>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +/* Initialize vbq */
>>>>> +vbq->type = node->desc.buf_type;
>>>>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
>>>>> +vbq->io_modes = VB2_MMAP;
>>>>> +else
>>>>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
>>>>> +
>>>>> +if (node->desc.smem_alloc) {
>>>>> +vbq->bidirectional = 1;
>>>>> +vbq->dev = cam->smem_dev;
>>>>> +} else {
>>>>> +vbq->dev = dev;
>>>>> +}
>>>>> +vbq->ops = &mtk_cam_vb2_ops;
>>>>> +vbq->mem_ops = &vb2_dma_contig_memops;
>>>>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
>>>>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
>>>>> +if (output)
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
>>>>> +else
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>>>>> +/* No minimum buffers limitation */
>>>>> +vbq->min_buffers_needed = 0;
>>>>> +vbq->drv_priv = cam;
>>>>> +vbq->lock = &node->vdev_lock;
>>>>> +vbq->supports_requests = true;
>>>>> +vbq->requires_requests = true;
>>>>> +
>>>>> +ret = vb2_queue_init(vbq);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
>>>>> +goto fail_media_clean;
>>>>> +}
>>>>> +
>>>>> +/* Initialize vdev */
>>>>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
>>>>> + dev_driver_string(dev), node->desc.name);
>>>>> +/* set cap/type/ioctl_ops of the video device */
>>>>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
>>>>> +vdev->ioctl_ops = node->desc.ioctl_ops;
>>>>> +vdev->fops = &mtk_cam_v4l2_fops;
>>>>> +vdev->release = video_device_release_empty;
>>>>> +vdev->lock = &node->vdev_lock;
>>>>> +vdev->v4l2_dev = &cam->v4l2_dev;
>>>>> +vdev->queue = &node->vbq;
>>>>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
>>>>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
>>>>> +vdev->entity.ops = NULL;
>>>>> +video_set_drvdata(vdev, cam);
>>>>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
>>>>> +
>>>>> +/* Initialize miscellaneous variables */
>>>>> +mutex_init(&node->vdev_lock);
>>>>> +INIT_LIST_HEAD(&node->buf_list);
>>>>> +spin_lock_init(&node->buf_list_lock);
>>>>> +
>>>>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register vde:%d\n", ret);
>>>>> +goto fail_vb2_rel;
>>>>> +}
>>>>> +
>>>>> +/* Create link between video node and the subdev pad */
>>>>> +if (output) {
>>>>> +ret = media_create_pad_link(&vdev->entity, 0,
>>>>> +    &cam->subdev.entity,
>>>>> +    node->id, link_flags);
>>>>> +} else {
>>>>> +ret = media_create_pad_link(&cam->subdev.entity,
>>>>> +    node->id, &vdev->entity, 0,
>>>>> +    link_flags);
>>>>> +}
>>>>
>>>> No need for the curly braces.
>>>>
>>>
>>> Revised in next patch.
>>>
>>>>> +if (ret)
>>>>> +goto fail_vdev_ureg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_ureg:
>>>>> +video_unregister_device(vdev);
>>>>> +fail_vb2_rel:
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +vb2_queue_release(vbq);
>>>>> +fail_media_clean:
>>>>> +media_entity_cleanup(&vdev->entity);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void
>>>>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +video_unregister_device(&node->vdev);
>>>>> +vb2_queue_release(&node->vbq);
>>>>> +media_entity_cleanup(&node->vdev.entity);
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +/* Set up media device & pads */
>>>>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
>>>>> +
>>>>> +/* Set up v4l2 device */
>>>>> +cam->v4l2_dev.mdev = &cam->media_dev;
>>>>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
>>>>> +
>>>>> +/* Initialize subdev */
>>>>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
>>>>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>>>>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
>>>>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>>>>> +V4L2_SUBDEV_FL_HAS_EVENTS;
>>>>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
>>>>> + "%s", dev_driver_string(dev));
>>>>> +v4l2_set_subdevdata(&cam->subdev, cam);
>>>>> +
>>>>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
>>>>> +goto fail_clean_media_entiy;
>>>>> +}
>>>>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
>>>>> +
>>>>> +/* Create video nodes and links */
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>>>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
>>>>> +
>>>>> +node->id = node->desc.id;
>>>>> +ret = mtk_cam_video_register_device(cam, node);
>>>>> +if (ret)
>>>>> +goto fail_vdev_unreg;
>>>>> +}
>>>>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_unreg:
>>>>> +for (i--; i >= 0; i--)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +fail_clean_media_entiy:
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int i;
>>>>> +
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +
>>>>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
>>>>> +v4l2_device_unregister_subdev(&cam->subdev);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
>>>>> +      struct v4l2_subdev *sd,
>>>>> +      struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>>>>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +cam->seninf = sd;
>>>>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
>>>>> +struct v4l2_subdev *sd,
>>>>> +struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +cam->seninf = NULL;
>>>>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!cam->seninf) {
>>>>> +dev_err(dev, "No seninf subdev\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
>>>>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
>>>>> +    MEDIA_LNK_FL_IMMUTABLE |
>>>>> +    MEDIA_LNK_FL_ENABLED);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
>>>>> +cam->seninf->entity.name, cam->subdev.entity.name,
>>>>> +ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
>>>>> +.bound = mtk_cam_dev_notifier_bound,
>>>>> +.unbind = mtk_cam_dev_notifier_unbind,
>>>>> +.complete = mtk_cam_dev_notifier_complete,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +v4l2_async_notifier_init(&cam->notifier);
>>>>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
>>>>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
>>>>
>>>> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
>>>>
>>>> Regards,
>>>> Helen
>>>>
>>>
>>> Ok, we will investigate how to do.
>>>
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
>>>>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
>>>>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +v4l2_async_notifier_unregister(&cam->notifier);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
>>>>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
>>>>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
>>>>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
>>>>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>>>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};> +
>>>>> +static const struct v4l2_format meta_fmts[] = {
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
>>>>> +.buffersize = 512 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_3A,
>>>>> +.buffersize = 1200 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_AF,
>>>>> +.buffersize = 640 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
>>>>> +.buffersize = 288 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
>>>>> +.buffersize = 256,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format stream_out_fmts[] = {
>>>>> +/* This is a default image format */
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format bin_out_fmts[] = {
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc output_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_IN_0,
>>>>> +.name = "meta input",
>>>>> +.cap = V4L2_CAP_META_OUTPUT,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = true,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 0,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc capture_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +.name = "main stream",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_IMGO,
>>>>> +.fmts = stream_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +.name = "packed out",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = 0,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_RRZO,
>>>>> +.fmts = bin_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_0,
>>>>> +.name = "partial meta 0",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AAO | R_FLKO | R_PSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 1,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_1,
>>>>> +.name = "partial meta 1",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AFO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 2,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_2,
>>>>> +.name = "partial meta 2",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LCSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 3,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_3,
>>>>> +.name = "partial meta 3",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LMVO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 4,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +/* The helper to configure the device context */
>>>>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +unsigned int node_idx;
>>>>> +int i;
>>>>> +
>>>>> +node_idx = 0;
>>>>> +/* Setup the output queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
>>>>> +
>>>>> +/* Setup the capture queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
>>>>> +}
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int ret;
>>>>> +
>>>>> +cam->dev = &pdev->dev;
>>>>> +mtk_cam_dev_queue_setup(cam);
>>>>> +
>>>>> +spin_lock_init(&cam->pending_job_lock);
>>>>> +spin_lock_init(&cam->running_job_lock);
>>>>> +INIT_LIST_HEAD(&cam->pending_job_list);
>>>>> +INIT_LIST_HEAD(&cam->running_job_list);
>>>>> +mutex_init(&cam->op_lock);
>>>>> +
>>>>> +/* v4l2 sub-device registration */
>>>>> +ret = mtk_cam_v4l2_register(cam);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +
>>>>> +ret = mtk_cam_v4l2_async_register(cam);
>>>>> +if (ret)
>>>>> +goto fail_v4l2_unreg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_v4l2_unreg:
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +mtk_cam_v4l2_async_unregister(cam);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +}
>>>>> +
>>>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> new file mode 100644
>>>>> index 000000000000..0a340a1e65ea
>>>>> --- /dev/null
>>>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> @@ -0,0 +1,244 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>>> +/*
>>>>> + * Copyright (c) 2019 MediaTek Inc.
>>>>> + */
>>>>> +
>>>>> +#ifndef __MTK_CAM_H__
>>>>> +#define __MTK_CAM_H__
>>>>> +
>>>>> +#include <linux/device.h>
>>>>> +#include <linux/types.h>
>>>>> +#include <linux/platform_device.h>
>>>>> +#include <linux/spinlock.h>
>>>>> +#include <linux/videodev2.h>
>>>>> +#include <media/v4l2-device.h>
>>>>> +#include <media/v4l2-ctrls.h>
>>>>> +#include <media/v4l2-subdev.h>
>>>>> +#include <media/videobuf2-core.h>
>>>>> +#include <media/videobuf2-v4l2.h>
>>>>> +
>>>>> +#include "mtk_cam-ipi.h"
>>>>> +
>>>>> +#define IMG_MAX_WIDTH5376
>>>>> +#define IMG_MAX_HEIGHT4032
>>>>> +#define IMG_MIN_WIDTH80
>>>>> +#define IMG_MIN_HEIGHT60
>>>>> +
>>>>> +/*
>>>>> + * ID enum value for struct mtk_cam_dev_node_desc:id
>>>>> + * or mtk_cam_video_device:id
>>>>> + */
>>>>> +enum  {
>>>>> +MTK_CAM_P1_META_IN_0 = 0,
>>>>> +MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +MTK_CAM_P1_META_OUT_0,
>>>>> +MTK_CAM_P1_META_OUT_1,
>>>>> +MTK_CAM_P1_META_OUT_2,
>>>>> +MTK_CAM_P1_META_OUT_3,
>>>>> +MTK_CAM_P1_TOTAL_NODES
>>>>> +};
>>>>> +
>>>>> +/* Supported image format list */
>>>>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
>>>>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
>>>>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
>>>>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
>>>>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
>>>>> +
>>>>> +/* Supported bayer pixel order */
>>>>> +#define MTK_CAM_RAW_PXL_ID_B0
>>>>> +#define MTK_CAM_RAW_PXL_ID_GB1
>>>>> +#define MTK_CAM_RAW_PXL_ID_GR2
>>>>> +#define MTK_CAM_RAW_PXL_ID_R3
>>>>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
>>>>> + *
>>>>> + * @frame_seq_no: The frame sequence of frame in driver layer.
>>>>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_p1_frame_param {
>>>>> +unsigned int frame_seq_no;
>>>>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_request - MTK camera device request.
>>>>> + *
>>>>> + * @req: Embedded struct media request.
>>>>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
>>>>> + * @frame_work: work queue entry for frame transmission to SCP.
>>>>> + * @list: List entry of the object for @struct mtk_cam_dev:
>>>>> + *        pending_job_list or running_job_list.
>>>>> + * @timestamp: Start of frame timestamp in ns
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_request {
>>>>> +struct media_request req;
>>>>> +struct mtk_p1_frame_param frame_params;
>>>>> +struct work_struct frame_work;
>>>>> +struct list_head list;
>>>>> +u64 timestamp;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
>>>>> + *
>>>>> + * @vbb: Embedded struct vb2_v4l2_buffer.
>>>>> + * @list: List entry of the object for @struct mtk_cam_video_device:
>>>>> + *        buf_list.
>>>>> + * @daddr: The DMA address of this buffer.
>>>>> + * @scp_addr: The SCP address of this buffer which
>>>>> + *            is only supported for meta input node.
>>>>> + * @node_id: The vidoe node id which this buffer belongs to.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_buffer {
>>>>> +struct vb2_v4l2_buffer vbb;
>>>>> +struct list_head list;
>>>>> +/* Intenal part */
>>>>> +dma_addr_t daddr;
>>>>> +dma_addr_t scp_addr;
>>>>> +unsigned int node_id;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
>>>>> + *
>>>>> + * @id: id of the node
>>>>> + * @name: name of the node
>>>>> + * @cap: supported V4L2 capabilities
>>>>> + * @buf_type: supported V4L2 buffer type
>>>>> + * @dma_port: the dma ports associated to the node
>>>>> + * @link_flags: default media link flags
>>>>> + * @smem_alloc: using the smem_dev as alloc device or not
>>>>> + * @image: true for image node, false for meta node
>>>>> + * @num_fmts: the number of supported node formats
>>>>> + * @default_fmt_idx: default format of this node
>>>>> + * @max_buf_count: maximum VB2 buffer count
>>>>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
>>>>> + * @fmts: supported format
>>>>> + * @frmsizes: supported V4L2 frame size number
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_node_desc {
>>>>> +u8 id;
>>>>> +const char *name;
>>>>> +u32 cap;
>>>>> +u32 buf_type;
>>>>> +u32 dma_port;
>>>>> +u32 link_flags;
>>>>> +u8 smem_alloc:1;
>>>>> +u8 image:1;
>>>>> +u8 num_fmts;
>>>>> +u8 default_fmt_idx;
>>>>> +u8 max_buf_count;
>>>>> +const struct v4l2_ioctl_ops *ioctl_ops;
>>>>> +const struct v4l2_format *fmts;
>>>>> +const struct v4l2_frmsizeenum *frmsizes;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_video_device - Mediatek video device structure
>>>>> + *
>>>>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
>>>>> + * @enabled: Indicate the video device is enabled or not
>>>>> + * @desc: The node description of video device
>>>>> + * @vdev_fmt: The V4L2 format of video device
>>>>> + * @vdev_pad: The media pad graph object of video device
>>>>> + * @vbq: A videobuf queue of video device
>>>>> + * @vdev: The video device instance
>>>>> + * @vdev_lock: Serializes vb2 queue and video device operations
>>>>> + * @buf_list: List for enqueue buffers
>>>>> + * @buf_list_lock: Lock used to protect buffer list.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_video_device {
>>>>> +unsigned int id;
>>>>> +unsigned int enabled;
>>>>> +struct mtk_cam_dev_node_desc desc;
>>>>> +struct v4l2_format vdev_fmt;
>>>>> +struct media_pad vdev_pad;
>>>>> +struct vb2_queue vbq;
>>>>> +struct video_device vdev;
>>>>> +/* Serializes vb2 queue and video device operations */
>>>>> +struct mutex vdev_lock;
>>>>> +struct list_head buf_list;
>>>>> +/* Lock used to protect buffer list */
>>>>> +spinlock_t buf_list_lock;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev - Mediatek camera device structure.
>>>>> + *
>>>>> + * @dev: Pointer to device.
>>>>> + * @smem_pdev: Pointer to shared memory device.
>>>>> + * @pipeline: Media pipeline information.
>>>>> + * @media_dev: Media device instance.
>>>>> + * @subdev: The V4L2 sub-device instance.
>>>>> + * @v4l2_dev: The V4L2 device driver instance.
>>>>> + * @notifier: The v4l2_device notifier data.
>>>>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
>>>>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
>>>>> + * @seninf: Pointer to the seninf sub-device.
>>>>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
>>>>> + * @streaming: Indicate the overall streaming status is on or off.
>>>>> + * @enabled_dmas: The enabled dma port information when streaming on.
>>>>> + * @enabled_count: Number of enabled video nodes
>>>>> + * @stream_count: Number of streaming video nodes
>>>>> + * @running_job_count: Nunber of running jobs in the HW driver.
>>>>> + * @pending_job_list: List to keep the media requests before en-queue into
>>>>> + *                    HW driver.
>>>>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
>>>>> + * @running_job_list: List to keep the media requests after en-queue into
>>>>> + *                    HW driver.
>>>>> + * @running_job_lock: Protect the running_job_list data.
>>>>> + * @op_lock: Serializes driver's VB2 callback operations.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev {
>>>>> +struct device *dev;
>>>>> +struct device *smem_dev;
>>>>> +struct media_pipeline pipeline;
>>>>> +struct media_device media_dev;
>>>>> +struct v4l2_subdev subdev;
>>>>> +struct v4l2_device v4l2_dev;
>>>>> +struct v4l2_async_notifier notifier;
>>>>> +struct media_pad *subdev_pads;
>>>>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
>>>>> +struct v4l2_subdev *seninf;
>>>>> +struct v4l2_subdev *sensor;
>>>>> +unsigned int streaming;
>>>>> +unsigned int enabled_dmas;
>>>>> +unsigned int enabled_count;
>>>>> +unsigned int stream_count;
>>>>> +unsigned int running_job_count;
>>>>> +struct list_head pending_job_list;
>>>>> +/* Protect the pending_job_list data */
>>>>> +spinlock_t pending_job_lock;
>>>>> +struct list_head running_job_list;
>>>>> +/* Protect the running_job_list data & running_job_count */
>>>>> +spinlock_t running_job_lock;
>>>>> +/* Serializes driver's VB2 callback operations */
>>>>> +struct mutex op_lock;
>>>>> +};
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
>>>>> +   unsigned int frame_seq_no);
>>>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
>>>>> +  unsigned int frame_seq_no);
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no);
>>>>> +
>>>>> +#endif /* __MTK_CAM_H__ */
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
>>>
>>
>> Regards,
>> Helen
> 
> Thanks for your comment.
> 
> Best regards,
> 
> 
> Jungo
> 

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver
@ 2020-05-05 15:38                 ` Helen Koike
  0 siblings, 0 replies; 388+ messages in thread
From: Helen Koike @ 2020-05-05 15:38 UTC (permalink / raw)
  To: Jungo Lin
  Cc: ryan.yu, frankie.chiu, laurent.pinchart, robh, suleiman,
	Jerry-ch.Chen, frederic.chen, linux-media, devicetree,
	hverkuil-cisco, sj.huang, yuzhao, linux-mediatek, Pi-Hsun Shih,
	matthias.bgg, mchehab, linux-arm-kernel, Sean.Cheng,
	srv_heupstream, shik, tfiga, zwisler, ddavenport



On 5/4/20 9:27 AM, Jungo Lin wrote:
> 
> Hi Helen;
> 
> Sorry for late reply.
> Please check my feedback & questions below.
> 
> On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
>> On 4/8/20 11:05 PM, Jungo Lin wrote:
>>> Hi Helen:
>>>
>>> Thanks for your comments.
>>>
>>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
>>>> Hello Jungo,
>>>>
>>>> I was taking a look at this patch (thanks for the work),
>>>> I didn't look in deep details, but I have some comments, please see
>>>> below. I hope it helps.
>>>>
>>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
>>>>> This patch adds the Mediatek ISP P1 HW control device driver.
>>>>> It handles the ISP HW configuration, provides interrupt handling and
>>>>> initializes the V4L2 device nodes and other V4L2 functions. Moreover,
>>>>> implement standard V4L2 video driver that utilizes V4L2 and media
>>>>> framework APIs. It supports one media device, one sub-device and
>>>>> several video devices during initialization. Moreover, it also connects
>>>>> with sensor and seninf drivers with V4L2 async APIs. Communicate with
>>>>> co-process via SCP communication to compose ISP registers in the
>>>>> firmware.
>>>>>
>>>>> (The current metadata interface used in meta input and partial
>>>>> meta nodes is only a temporary solution to kick off the driver
>>>>> development and is not ready to be reviewed yet.)
>>>>>
>>>>> Signed-off-by: Jungo Lin <jungo.lin@mediatek.com>
>>>>> Signed-off-by: Tomasz Figa <tfiga@chromium.org>
>>>>> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
>>>>> ---
>>>>> Changes from v6:
>>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1
>>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih
>>>>>  - Correct auto suspend timer value for suspend/resume issue
>>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command timeout event
>>>>>  - Fix KE due to no sen-inf sub-device
>>>>> ---
>>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
>>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
>>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
>>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
>>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
>>>>
>>>> I think I would split this file a bit, to separate which code is being used for the subdevice, which for
>>>> capture, which for metadata, and what is being used to deal with requests.
>>>>
>>>> It would make it easier to review imho.
>>>>
>>>
>>> For file structure design, it was reviewed in the previous patch
>>> serials.
>>> e.g.
>>> https://patchwork.kernel.org/patch/10938137/
>>> If you think it is better, I will modify it.
>>
>> Right, I saw a suggestion to merge two files there.
>>
>> I'm not sure what others think, but I'm used to see a separation per entity, or at least separate subdevices
>> from video devices, it is easier to see which v4l2 functions is being called per entity IMHO.
>> So it reflects a bit the topology.
>> But it is also up to you to see if it improves organization or not, it is just a suggestion.
>>
> 
> Ok, got your point.
> We will discuss how to do internally.
> 
> [snip]
>>>>> +isp_composer_hw_init(p1_dev);
>>>>> +
>>>>> +p1_dev->enqueued_frame_seq_no = 0;
>>>>> +p1_dev->dequeued_frame_seq_no = 0;
>>>>> +p1_dev->composed_frame_seq_no = 0;
>>>>> +p1_dev->sof_count = 0;
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +int mtk_isp_hw_release(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(dev);
>>>>> +
>>>>> +isp_composer_hw_deinit(p1_dev);
>>>>> +pm_runtime_mark_last_busy(dev);
>>>>> +pm_runtime_put_autosuspend(dev);
>>>>> +rproc_shutdown(p1_dev->rproc_handle);
>>>>> +
>>>>> +dev_dbg(dev, "%s done\n", __func__);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +void mtk_isp_req_enqueue(struct mtk_cam_dev *cam,
>>>>> + struct mtk_cam_dev_request *req)
>>>>> +{
>>>>> +struct mtk_isp_p1_device *p1_dev = dev_get_drvdata(cam->dev);
>>>>> +
>>>>> +/* Accumulated frame sequence number */
>>>>> +req->frame_params.frame_seq_no = ++p1_dev->enqueued_frame_seq_no;
>>>>> +
>>>>> +INIT_WORK(&req->frame_work, isp_tx_frame_worker);
>>>>> +queue_work(p1_dev->composer_wq, &req->frame_work);
>>>>> +dev_dbg(cam->dev, "enqueue fd:%s frame_seq_no:%d job cnt:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no,
>>>>> +cam->running_job_count);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_sof(struct mtk_isp_p1_device *p1_dev,
>>>>> +       unsigned int dequeued_frame_seq_no)
>>>>> +{
>>>>> +dma_addr_t base_addr = p1_dev->composer_iova;
>>>>> +struct device *dev = p1_dev->dev;
>>>>> +struct mtk_cam_dev_request *req;
>>>>> +int composed_frame_seq_no = p1_dev->composed_frame_seq_no;
>>>>> +unsigned int addr_offset;
>>>>> +
>>>>> +/* Send V4L2_EVENT_FRAME_SYNC event */
>>>>> +mtk_cam_dev_event_frame_sync(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +
>>>>> +p1_dev->sof_count += 1;
>>>>> +/* Save frame information */
>>>>> +p1_dev->dequeued_frame_seq_no = dequeued_frame_seq_no;
>>>>> +
>>>>> +req = mtk_cam_dev_get_req(&p1_dev->cam_dev, dequeued_frame_seq_no);
>>>>> +if (req)
>>>>> +req->timestamp = ktime_get_boottime_ns();
>>>>> +
>>>>> +/* Update CQ base address if needed */
>>>>> +if (composed_frame_seq_no <= dequeued_frame_seq_no) {
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, no update, cq_num:%d, frame_seq:%d\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no);
>>>>> +return;
>>>>> +}
>>>>> +addr_offset = MTK_ISP_CQ_ADDRESS_OFFSET *
>>>>> +(dequeued_frame_seq_no % MTK_ISP_CQ_BUFFER_COUNT);
>>>>> +writel(base_addr + addr_offset, p1_dev->regs + REG_CQ_THR0_BASEADDR);
>>>>> +dev_dbg(dev,
>>>>> +"SOF_INT_ST, update next, cq_num:%d, frame_seq:%d cq_addr:0x%x\n",
>>>>> +composed_frame_seq_no, dequeued_frame_seq_no, addr_offset);
>>>>> +}
>>>>> +
>>>>> +static void isp_irq_handle_dma_err(struct mtk_isp_p1_device *p1_dev)
>>>>> +{
>>>>> +u32 val;
>>>>> +
>>>>> +dev_err(p1_dev->dev,
>>>>> +"IMGO:0x%x, RRZO:0x%x, AAO=0x%x, AFO=0x%x, LMVO=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_IMGO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_RRZO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AAO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_AFO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LMVO_ERR_STAT));
>>>>> +dev_err(p1_dev->dev,
>>>>> +"LCSO=0x%x, PSO=0x%x, FLKO=0x%x, BPCI:0x%x, LSCI=0x%x\n",
>>>>> +readl(p1_dev->regs + REG_LCSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_PSO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_FLKO_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_BPCI_ERR_STAT),
>>>>> +readl(p1_dev->regs + REG_LSCI_ERR_STAT));
>>>>
>>>> I think if would be better to transfor those into dev_dbg and add a counter
>>>> in debugfs.
>>>>
>>>
>>> These error messages are important for debugging.
>>> I suggest to keep in dev_err.
>>
>> I mean, these messages are usefull for debug (as you mentioned yourself), but for an
>> end user not so much, since end users won't know the meaning of those values.
>>
>> For end users a "dma failure" message would be enough, then advanced users can enable
>> debug messages to see more.
> 
> OK.
> Got your point.
> 
>>>
>>> Moreover, could you give more information about debug counter?
>>> I don't get your point.
>>> Do you suggest to accumulate the total count of DMA errors?
>>
>>
>> Yes, you could have a debugfs entry with error counters like:
>>
>> cat /debugfs/mtk_isp/dma_err
>> 8
>>
>> So it is easier if this error happens very frequent or not.
>> In the rkisp1 case we added:
>>
>> /debugfs/rkisp1/data_loss
>> /debugfs/rkisp1/pic_size_error
>> /debugfs/rkisp1/mipi_error
>> /debugfs/rkisp1/stats_error
>> /debugfs/rkisp1/mp_stop_timeout
>> /debugfs/rkisp1/sp_stop_timeout
>> /debugfs/rkisp1/mp_frame_drop
>> /debugfs/rkisp1/sp_frame_drop
>>
>> Also, these error are non fatal, userspace can continue to work (in a way) when they happen,
>> so the idea was not to flood the logs with messages that end users don't care much, if they are frequent.
>>
>> But I'm not sure if this applies well or if it is useful to you case (please don't take my suggestions blindly).
>>
> 
> Ok, we will follow your suggestion.
> 
> 
> [snip]
> 
>>>>> +return;
>>>>> +
>>>>> +dev_dbg(cam->dev, "job done request:%s frame_seq:%d state:%d\n",
>>>>> +req->req.debug_str, req->frame_params.frame_seq_no, state);
>>>>> +
>>>>> +list_for_each_entry_safe(obj, obj_prev, &req->req.objects, list) {
>>>>> +struct vb2_buffer *vb;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct mtk_cam_video_device *node;
>>>>> +
>>>>> +if (!vb2_request_object_is_buffer(obj))
>>>>> +continue;
>>>>> +vb = container_of(obj, struct vb2_buffer, req_obj);
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_del(&buf->list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +buf->vbb.sequence = req->frame_params.frame_seq_no;
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
>>>>> +vb->timestamp = ts_eof;
>>>>> +else
>>>>> +vb->timestamp = req->timestamp;
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, get frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +return req;
>>>>> +}
>>>>> +}
>>>>> +spin_unlock_irqrestore(&cam->running_job_lock, flags);
>>>>> +
>>>>> +return NULL;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam,
>>>>> +   unsigned int frame_seq_no)
>>>>> +{
>>>>> +struct mtk_cam_dev_request *req, *req_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&cam->running_job_lock, flags);
>>>>> +list_for_each_entry_safe(req, req_prev, &cam->running_job_list, list) {
>>>>> +dev_dbg(cam->dev, "frame_seq:%d, de-queue frame_seq:%d\n",
>>>>> +req->frame_params.frame_seq_no, frame_seq_no);
>>>>> +
>>>>> +/* Match by the en-queued request number */
>>>>> +if (req->frame_params.frame_seq_no == frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_DONE);
>>>>> +list_del(&req->list);
>>>>> +break;
>>>>> +} else if (req->frame_params.frame_seq_no < frame_seq_no) {
>>>>> +cam->running_job_count--;
>>>>> +/* Pass to user space for frame drop */
>>>>> +mtk_cam_dev_job_done(cam, req, VB2_BUF_STATE_ERROR);
>>>>> +dev_warn(cam->dev, "frame_seq:%d drop\n",
>>>>> + req->frame_params.frame_seq_no);
>>>>
>>>> maybe a counter in debugfs instead of the warning.
>>>>
>>>
>>> Do you mean to add counter to accumulate the total count of drop frames?
>>
>> please see my comment above.
>>
> 
> Ok, add this in next patch.
> 
>>> Could we add this and also keep this warning message?
>>
>> Userspace would still continue to work when this happens, not sure if it is worthy
>> adding a warn, I would move it to dev_dbg() instead IMHO.
>>
> 
> Ok, revise in next patch.
> 
> [snip]
>>>>> +
>>>>> +static void cal_image_pix_mp(struct mtk_cam_dev *cam, unsigned int node_id,
>>>>> +     struct v4l2_pix_format_mplane *mp)
>>>>> +{
>>>>> +unsigned int bpl, ppl;
>>>>
>>>> bytes per line and pixels per line right?
>>>>
>>>
>>> Yes.
>>>
>>>>> +unsigned int pixel_bits = get_pixel_bits(mp->pixelformat);
>>>>
>>>> wouldn't be easier a get_pixel_bytes() function instead of bits?
>>>>
>>>
>>> Sorry. I didn't get the point.
>>> The unit of return value is bits, not bytes.
>>> Do you suggest move bpl & ppl calculation into get_pixel_bits() and
>>> rename to get_pixel_bytes()?
>>
>> Never mind, I misread it.
>>
> 
> Ok, we will skip this.
> 
> [snip]
>>>>> +unsigned int enabled_dma_ports = cam->enabled_dmas;
>>>>> +int ret;
>>>>> +
>>>>> +/* Get sensor format configuration */
>>>>> +sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>>>>> +ret = v4l2_subdev_call(cam->sensor, pad, get_fmt, NULL, &sd_fmt);
>>>>> +if (ret) {
>>>>> +dev_dbg(dev, "sensor g_fmt failed:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +sd_width = sd_fmt.format.width;
>>>>> +sd_height = sd_fmt.format.height;
>>>>> +sd_code = sd_fmt.format.code;
>>>>> +dev_dbg(dev, "sd fmt w*h=%d*%d, code=0x%x\n", sd_width, sd_height,
>>>>> +sd_code);
>>>>
>>>> If V4L2_SUBDEV_FL_HAS_DEVNODE is used, then format shouldn't propagate from one node to the other,
>>>> it should be configured from userspace.
>>>>
>>>
>>> Could you explain why?
>>> Moreover, how does configuration from user space?
>>
>> IIUC there are two ways to configure the topology, see Hans comment on https://lkml.org/lkml/2020/2/6/305
>>
>> If you use v4l2_device_register_subdev_nodes(), it exposes a /dev/v4l-subdevX file to userspace
>> in all subdevices you have the flag V4L2_SUBDEV_FL_HAS_DEVNODE (and you have it in the isp node).
>>
>> Which means that if the sensor implements VIDIOC_SUBDEV_S_FMT, part of the subdevices in the topology
>> can be configured by userspace and part can't (which iirc should't be done in the media API).
>>
>> Do you need to use v4l2_device_register_subdev_nodes() ?
>>
>> Also, Jacopo's patchset introduces a v4l2_device_register_ro_subdev_nodes() fuction:
>> https://patchwork.kernel.org/cover/11463183/
>>
>> which would be more appropriated if you don't want userspace to configure the whole pipeline.
>>
> 
> The purpose of P1 sub-device is to provide V4L2 event subscribe with
> V4L2_EVENT_FRAME_SYNC event for user space. It is the same
> implementation as blow link.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/omap3isp/ispccdc.c#L1853
> 
> As you suggest, we may use v4l2_device_register_ro_subdev_nodes() more
> precisely.
> 
> [snip]
> 
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +struct mtk_cam_dev_request *req = mtk_cam_req_to_dev_req(vb->request);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +dev_dbg(dev, "%s: node:%d fd:%d idx:%d\n", __func__,
>>>>> +node->id, buf->vbb.request_fd, buf->vbb.vb2_buf.index);
>>>>> +
>>>>> +/* added the buffer into the tracking list */
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_add_tail(&buf->list, &node->buf_list);
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +
>>>>> +/* update buffer internal address */
>>>>> +req->frame_params.dma_bufs[buf->node_id].iova = buf->daddr;
>>>>> +req->frame_params.dma_bufs[buf->node_id].scp_addr = buf->scp_addr;
>>>>
>>>> isn't it an issue if userspace queue two buffers for the same video device in the same request?
>>>>
>>>> vb2_request_queue(req) will call all the .buf_queue() callbacks, and only the last buffer in the list
>>>> will be at req->frame_params.dma_bufs[buf->node_id], no?
>>>>
>>>> Also, what happens if a request doesn't contain buffers for all node_ids ? Will it put data in the previous programmed
>>>> buffer?
>>>>
>>>> Please, let me know if these questions doesn't make sense, I'm not that familiar with the request API internals.
>>>>
>>>
>>> 1. yes, it is a issue if userspace queues two buffers for the same video
>>> device with the same request FD.
>>>
>>> 2. All buffers which are belonged different to different video devices
>>> in the request list will be updated to req->frame_params.dma_bufs by
>>> buf->node_id.
>>>
>>> 3. It is not allowed for userspace to queue partial buffers for all
>>> enabled video devices. If it happens, it may trigger DMA errors for this
>>> request.
>>
>> So I guess you should implement a custom .req_validate() to enforce userspace follows this.
>>
> 
> For case 1, it is handled in the vb2_queue_or_prepare_buf.
> https://elixir.bootlin.com/linux/latest/source/drivers/media/common/videobuf2/videobuf2-v4l2.c#L453

Ok, thanks for the link.

> 
> For case 3, I need to correct my previous answer. This behavior should
> be OK for outputted DMA. We have frame buffer controller for each
> outputted DMAs. So it means the tuning buffer node is mandatory for each
> request, other nodes are optional. We will implement this
> in .req_validate to check.
> 
>>>
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct device *dev = cam->dev;
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +dma_addr_t addr;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +buf->node_id = node->id;
>>>>> +buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
>>>>> +buf->scp_addr = 0;
>>>>> +
>>>>> +/* SCP address is only valid for meta input buffer */
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return 0;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +/* Use coherent address to get iova address */
>>>>> +addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
>>>>> +DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);> +if (dma_mapping_error(dev, addr)) {
>>>>> +dev_err(dev, "failed to map meta addr:%pad\n", &buf->daddr);
>>>>> +return -EFAULT;
>>>>> +}
>>>>> +buf->scp_addr = buf->daddr;
>>>>> +buf->daddr = addr;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +if (vb2_plane_size(vb, 0) < size) {
>>>>> +dev_dbg(cam->dev, "plane size is too small:%lu<%u\n",
>>>>> +vb2_plane_size(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +
>>>>> +if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
>>>>> +if (vb2_get_plane_payload(vb, 0) != size) {
>>>>> +dev_dbg(cam->dev, "plane payload is mismatch:%lu:%u\n",
>>>>> +vb2_get_plane_payload(vb, 0), size);
>>>>> +return -EINVAL;
>>>>> +}
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +v4l2_buf->field = V4L2_FIELD_NONE;
>>>>> +vb2_set_plane_payload(vb, 0, size);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +struct mtk_cam_dev_buffer *buf;
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +if (!node->desc.smem_alloc)
>>>>> +return;
>>>>> +
>>>>> +buf = mtk_cam_vb2_buf_to_dev_buf(vb);
>>>>> +dma_unmap_page_attrs(dev, buf->daddr,
>>>>> +     vb->planes[0].length,
>>>>> +     DMA_BIDIRECTIONAL,
>>>>> +     DMA_ATTR_SKIP_CPU_SYNC);
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vb->vb2_queue);
>>>>> +
>>>>> +dev_dbg(cam->dev, "%s\n", __func__);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
>>>>> +   unsigned int *num_buffers,
>>>>> +   unsigned int *num_planes,
>>>>> +   unsigned int sizes[],
>>>>> +   struct device *alloc_devs[])
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +unsigned int max_buffer_count = node->desc.max_buf_count;
>>>>> +const struct v4l2_format *fmt = &node->vdev_fmt;
>>>>> +unsigned int size;
>>>>> +
>>>>> +/* Check the limitation of buffer size */
>>>>> +if (max_buffer_count)
>>>>> +*num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
>>>>> +
>>>>> +if (node->desc.smem_alloc)
>>>>> +vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>>>>> +
>>>>> +if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
>>>>> +    vq->type == V4L2_BUF_TYPE_META_CAPTURE)
>>>>> +size = fmt->fmt.meta.buffersize;
>>>>> +else
>>>>> +size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
>>>>> +
>>>>> +/* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
>>>>> +if (*num_planes) {
>>>>> +if (sizes[0] < size || *num_planes != 1)
>>>>> +return -EINVAL;
>>>>> +} else {
>>>>> +*num_planes = 1;
>>>>> +sizes[0] = size;
>>>>> +}
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_return_all_buffers(struct mtk_cam_dev *cam,
>>>>> +   struct mtk_cam_video_device *node,
>>>>> +   enum vb2_buffer_state state)
>>>>> +{
>>>>> +struct mtk_cam_dev_buffer *buf, *buf_prev;
>>>>> +unsigned long flags;
>>>>> +
>>>>> +spin_lock_irqsave(&node->buf_list_lock, flags);
>>>>> +list_for_each_entry_safe(buf, buf_prev, &node->buf_list, list) {
>>>>> +list_del(&buf->list);
>>>>> +vb2_buffer_done(&buf->vbb.vb2_buf, state);
>>>>> +}
>>>>> +spin_unlock_irqrestore(&node->buf_list_lock, flags);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
>>>>> +       unsigned int count)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!node->enabled) {
>>>>> +dev_err(dev, "Node:%d is not enabled\n", node->id);
>>>>> +ret = -ENOLINK;
>>>>> +goto fail_ret_buf;
>>>>> +}
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +/* Start streaming of the whole pipeline now*/
>>>>> +if (!cam->pipeline.streaming_count) {
>>>>
>>>> No need for this check, vb2 won't call .start_streaming() twice without stop_streaming() in between.
>>>>
>>>
>>> The check is designed to start the media pipeline when we start
>>> streaming on the first node. You could refer the detail in below link.
>>>
>>> https://patchwork.kernel.org/patch/10985819/
>>
>> right, ok, this is when enabling streaming from multiple nodes.
>>
>> media_pipeline_start() is usually called for every stream that starts.
>>
>> So cam->pipeline.streaming_count can reflect the number of streams enabled.
>>
>> So maybe you don't need cam->stream_count.
>>
> 
> Ok, revise in next patch.
> 
>>>
>>>
>>>>> +ret = media_pipeline_start(&node->vdev.entity, &cam->pipeline);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to start pipeline:%d\n", ret);
>>>>> +goto fail_unlock;
>>>>> +}
>>>>> +mtk_cam_dev_init_stream(cam);
>>>>> +ret = mtk_isp_hw_init(cam);
>>
>> Would it make sense to move this to s_stream in the ISP's subdevice ?
>>
> 
> No, we like to initialize our ISP firmware here when the first video
> node is streaming on. It is too late to initialize when all video nodes
> are streaming-on.
> 
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init HW:%d\n", ret);
>>>>> +goto fail_stop_pipeline;
>>>>> +}
>>>>> +}
>>>>> +
>>>>> +/* Media links are fixed after media_pipeline_start */
>>>>> +cam->stream_count++;
>>>>> +dev_dbg(dev, "%s: count info:%d:%d\n", __func__, cam->stream_count,
>>>>> +cam->enabled_count);
>>>>> +if (cam->stream_count < cam->enabled_count) {
>>
>> I'm also wondering, since you need to wait for all the enabled video devices
>> to start streaming, shouldn't this be done inside a request? So you can enable
>> all of them at once?
>>
>> Also, like this you wouldn't need to check enabled links to query for enabled video
>> nodes, you can just enable the ones in the request.
>>
>> make sense?
>>
> 
> Sorry, I didn't get your point about this comment.
> Which request could we handle this logic?
> Do you mean move this logic into mtk_cam_req_queue function?

Sorry me, I thought we could use request api to call VIDIOC_STREAMON for multiple capture nodes,
but it seems we can't (please see my reply on the cover letter).

Regards,
Helen

> 
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +/* Stream on sub-devices node */
>>>>> +ret = v4l2_subdev_call(&cam->subdev, video, s_stream, 1);
>>>>> +if (ret)
>>>>> +goto fail_no_stream;
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_no_stream:
>>>>> +cam->stream_count--;
>>>>> +fail_stop_pipeline:
>>>>> +if (cam->stream_count == 0)
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +fail_unlock:
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +fail_ret_buf:
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_QUEUED);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = vb2_get_drv_priv(vq);
>>>>> +struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
>>>>> +struct device *dev = cam->dev;
>>>>> +
>>>>> +mutex_lock(&cam->op_lock);
>>>>> +dev_dbg(dev, "%s node:%d count info:%d\n", __func__, node->id,
>>>>> +cam->stream_count);
>>>>> +/* Check the first node to stream-off */
>>>>> +if (cam->stream_count == cam->enabled_count)
>>>>> +v4l2_subdev_call(&cam->subdev, video, s_stream, 0);
>>>>> +
>>>>> +mtk_cam_vb2_return_all_buffers(cam, node, VB2_BUF_STATE_ERROR);
>>>>> +cam->stream_count--;
>>>>> +if (cam->stream_count) {
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +return;
>>>>> +}
>>>>> +mutex_unlock(&cam->op_lock);
>>>>> +
>>>>> +mtk_cam_dev_req_cleanup(cam);
>>>>> +media_pipeline_stop(&node->vdev.entity);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_querycap(struct file *file, void *fh,
>>>>> +   struct v4l2_capability *cap)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +
>>>>> +strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
>>>>> +strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
>>>>> +snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
>>>>> + dev_name(cam->dev));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
>>>>> +   struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index >= node->desc.num_fmts)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->desc.fmts[f->index].fmt.pix_mp.pixelformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt = node->vdev_fmt.fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
>>>>> +  struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +struct device *dev = cam->dev;
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +struct v4l2_format try_fmt;
>>>>> +
>>>>> +memset(&try_fmt, 0, sizeof(try_fmt));
>>>>> +try_fmt.type = f->type;
>>>>> +
>>>>> +/* Validate pixelformat */
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
>>>>> +if (!dev_fmt) {
>>>>> +dev_dbg(dev, "unknown fmt:%d\n", f->fmt.pix_mp.pixelformat);
>>>>> +dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx];
>>>>> +}
>>>>> +try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
>>>>> +
>>>>> +/* Validate image width & height range */
>>>>> +try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
>>>>> +     IMG_MIN_WIDTH, IMG_MAX_WIDTH);
>>>>> +try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
>>>>> +      IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
>>>>> +/* 4 bytes alignment for width */
>>>>> +try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, 4);
>>>>> +
>>>>> +/* Only support one plane */
>>>>> +try_fmt.fmt.pix_mp.num_planes = 1;
>>>>> +
>>>>> +/* bytesperline & sizeimage calculation */
>>>>> +cal_image_pix_mp(cam, node->id, &try_fmt.fmt.pix_mp);
>>>>> +
>>>>> +/* Constant format fields */
>>>>> +try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
>>>>> +try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
>>>>> +try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>>> +try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
>>>>> +
>>>>> +*f = try_fmt;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam = video_drvdata(file);
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (vb2_is_busy(node->vdev.queue)) {
>>>>> +dev_dbg(cam->dev, "%s: queue is busy\n", __func__);
>>>>> +return -EBUSY;
>>>>> +}
>>>>> +
>>>>> +/* Get the valid format */
>>>>> +mtk_cam_vidioc_try_fmt(file, fh, f);
>>>>> +/* Configure to video device */
>>>>> +node->vdev_fmt = *f;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
>>>>> +  struct v4l2_frmsizeenum *sizes)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
>>>>> +const struct v4l2_format *dev_fmt;
>>>>> +
>>>>> +dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
>>>>> +if (!dev_fmt || sizes->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +sizes->type = node->desc.frmsizes->type;
>>>>> +memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
>>>>> +       sizeof(sizes->stepwise));
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
>>>>> +struct v4l2_fmtdesc *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +if (f->index)
>>>>> +return -EINVAL;
>>>>> +
>>>>> +/* f->description is filled in v4l_fill_fmtdesc function */
>>>>> +f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->flags = 0;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
>>>>> +     struct v4l2_format *f)
>>>>> +{
>>>>> +struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
>>>>> +
>>>>> +f->fmt.meta.dataformat = node->vdev_fmt.fmt.meta.dataformat;
>>>>> +f->fmt.meta.buffersize = node->vdev_fmt.fmt.meta.buffersize;
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_subdev_core_ops mtk_cam_subdev_core_ops = {
>>>>> +.subscribe_event = mtk_cam_sd_subscribe_event,
>>>>> +.unsubscribe_event = v4l2_event_subdev_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_video_ops mtk_cam_subdev_video_ops = {
>>>>> +.s_stream =  mtk_cam_sd_s_stream,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_subdev_ops mtk_cam_subdev_ops = {
>>>>> +.core = &mtk_cam_subdev_core_ops,
>>>>> +.video = &mtk_cam_subdev_video_ops,
>>>>> +};
>>>>
>>>> hmm, since this subdevice is exposed with V4L2_SUBDEV_FL_HAS_DEVNODE,
>>>> I wonder if pad ops shouldn't be implemented too (to be verified).
>>>>
>>>
>>> Ok, I will investigate this.
>>>
>>>>> +
>>>>> +static const struct media_entity_operations mtk_cam_media_entity_ops = {
>>>>> +.link_setup = mtk_cam_media_link_setup,
>>>>> +.link_validate = v4l2_subdev_link_validate,
>>>>> +};
>>>>> +
>>>>> +static const struct vb2_ops mtk_cam_vb2_ops = {
>>>>> +.queue_setup = mtk_cam_vb2_queue_setup,
>>>>> +.wait_prepare = vb2_ops_wait_prepare,
>>>>> +.wait_finish = vb2_ops_wait_finish,
>>>>> +.buf_init = mtk_cam_vb2_buf_init,
>>>>> +.buf_prepare = mtk_cam_vb2_buf_prepare,
>>>>> +.start_streaming = mtk_cam_vb2_start_streaming,
>>>>> +.stop_streaming = mtk_cam_vb2_stop_streaming,
>>>>> +.buf_queue = mtk_cam_vb2_buf_queue,
>>>>> +.buf_cleanup = mtk_cam_vb2_buf_cleanup,
>>>>> +.buf_request_complete = mtk_cam_vb2_request_complete,
>>>>> +};> +
>>>>> +static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
>>>>> +.unlocked_ioctl = video_ioctl2,
>>>>> +.open = v4l2_fh_open,
>>>>> +.release = vb2_fop_release,
>>>>> +.poll = vb2_fop_poll,
>>>>> +.mmap = vb2_fop_mmap,
>>>>> +#ifdef CONFIG_COMPAT
>>>>> +.compat_ioctl32 = v4l2_compat_ioctl32,
>>>>> +#endif
>>>>> +};
>>>>> +
>>>>> +static const struct media_device_ops mtk_cam_media_ops = {
>>>>> +.req_alloc = mtk_cam_req_alloc,
>>>>> +.req_free = mtk_cam_req_free,
>>>>> +.req_validate = vb2_request_validate,
>>>>> +.req_queue = mtk_cam_req_queue,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_media_register(struct mtk_cam_dev *cam,
>>>>> +  struct media_device *media_dev)
>>>>> +{
>>>>> +/* Reserved MTK_CAM_CIO_PAD_SINK + 1 pads to use */
>>>>> +unsigned int num_pads = MTK_CAM_CIO_PAD_SINK + 1;
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +media_dev->dev = cam->dev;
>>>>> +strscpy(media_dev->model, dev_driver_string(dev),
>>>>> +sizeof(media_dev->model));
>>>>> +snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
>>>>> + "platform:%s", dev_name(dev));
>>>>> +media_dev->hw_revision = 0;
>>>>> +media_device_init(media_dev);
>>>>> +media_dev->ops = &mtk_cam_media_ops;
>>>>> +
>>>>> +ret = media_device_register(media_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register media device:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +/* Initialize subdev pads */
>>>>> +cam->subdev_pads = devm_kcalloc(dev, num_pads,
>>>>> +sizeof(*cam->subdev_pads),
>>>>> +GFP_KERNEL);
>>>>> +if (!cam->subdev_pads) {
>>>>> +dev_err(dev, "failed to allocate subdev_pads\n");
>>>>> +ret = -ENOMEM;
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +ret = media_entity_pads_init(&cam->subdev.entity, num_pads,
>>>>> +     cam->subdev_pads);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pads:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +
>>>>> +/* Initialize all pads with MEDIA_PAD_FL_SOURCE */
>>>>> +for (i = 0; i < num_pads; i++)
>>>>> +cam->subdev_pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Customize the last one pad as CIO sink pad. */
>>>>> +cam->subdev_pads[MTK_CAM_CIO_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int
>>>>> +mtk_cam_video_register_device(struct mtk_cam_dev *cam,
>>>>> +      struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +struct video_device *vdev = &node->vdev;
>>>>> +struct vb2_queue *vbq = &node->vbq;
>>>>> +unsigned int output = V4L2_TYPE_IS_OUTPUT(node->desc.buf_type);
>>>>> +unsigned int link_flags = node->desc.link_flags;
>>>>> +int ret;
>>>>> +
>>>>> +/* Initialize mtk_cam_video_device */
>>>>> +if (link_flags & MEDIA_LNK_FL_IMMUTABLE)
>>>>> +node->enabled = true;
>>>>> +else
>>>>> +node->enabled = false;
>>>>> +mtk_cam_dev_load_default_fmt(cam, &node->desc, &node->vdev_fmt);
>>>>> +
>>>>> +cam->subdev_pads[node->id].flags = output ?
>>>>> +MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>>>> +
>>>>> +/* Initialize media entities */
>>>>> +ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize media pad:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +node->vdev_pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>>>> +
>>>>> +/* Initialize vbq */
>>>>> +vbq->type = node->desc.buf_type;
>>>>> +if (vbq->type == V4L2_BUF_TYPE_META_OUTPUT)
>>>>> +vbq->io_modes = VB2_MMAP;
>>>>> +else
>>>>> +vbq->io_modes = VB2_MMAP | VB2_DMABUF;
>>>>> +
>>>>> +if (node->desc.smem_alloc) {
>>>>> +vbq->bidirectional = 1;
>>>>> +vbq->dev = cam->smem_dev;
>>>>> +} else {
>>>>> +vbq->dev = dev;
>>>>> +}
>>>>> +vbq->ops = &mtk_cam_vb2_ops;
>>>>> +vbq->mem_ops = &vb2_dma_contig_memops;
>>>>> +vbq->buf_struct_size = sizeof(struct mtk_cam_dev_buffer);
>>>>> +vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_BOOTIME;
>>>>> +if (output)
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
>>>>> +else
>>>>> +vbq->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>>>>> +/* No minimum buffers limitation */
>>>>> +vbq->min_buffers_needed = 0;
>>>>> +vbq->drv_priv = cam;
>>>>> +vbq->lock = &node->vdev_lock;
>>>>> +vbq->supports_requests = true;
>>>>> +vbq->requires_requests = true;
>>>>> +
>>>>> +ret = vb2_queue_init(vbq);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to init. vb2 queue:%d\n", ret);
>>>>> +goto fail_media_clean;
>>>>> +}
>>>>> +
>>>>> +/* Initialize vdev */
>>>>> +snprintf(vdev->name, sizeof(vdev->name), "%s %s",
>>>>> + dev_driver_string(dev), node->desc.name);
>>>>> +/* set cap/type/ioctl_ops of the video device */
>>>>> +vdev->device_caps = node->desc.cap | V4L2_CAP_STREAMING;
>>>>> +vdev->ioctl_ops = node->desc.ioctl_ops;
>>>>> +vdev->fops = &mtk_cam_v4l2_fops;
>>>>> +vdev->release = video_device_release_empty;
>>>>> +vdev->lock = &node->vdev_lock;
>>>>> +vdev->v4l2_dev = &cam->v4l2_dev;
>>>>> +vdev->queue = &node->vbq;
>>>>> +vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
>>>>> +vdev->entity.function = MEDIA_ENT_F_IO_V4L;
>>>>> +vdev->entity.ops = NULL;
>>>>> +video_set_drvdata(vdev, cam);
>>>>> +dev_dbg(dev, "registered vdev:%d:%s\n", node->id, vdev->name);
>>>>> +
>>>>> +/* Initialize miscellaneous variables */
>>>>> +mutex_init(&node->vdev_lock);
>>>>> +INIT_LIST_HEAD(&node->buf_list);
>>>>> +spin_lock_init(&node->buf_list_lock);
>>>>> +
>>>>> +ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register vde:%d\n", ret);
>>>>> +goto fail_vb2_rel;
>>>>> +}
>>>>> +
>>>>> +/* Create link between video node and the subdev pad */
>>>>> +if (output) {
>>>>> +ret = media_create_pad_link(&vdev->entity, 0,
>>>>> +    &cam->subdev.entity,
>>>>> +    node->id, link_flags);
>>>>> +} else {
>>>>> +ret = media_create_pad_link(&cam->subdev.entity,
>>>>> +    node->id, &vdev->entity, 0,
>>>>> +    link_flags);
>>>>> +}
>>>>
>>>> No need for the curly braces.
>>>>
>>>
>>> Revised in next patch.
>>>
>>>>> +if (ret)
>>>>> +goto fail_vdev_ureg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_ureg:
>>>>> +video_unregister_device(vdev);
>>>>> +fail_vb2_rel:
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +vb2_queue_release(vbq);
>>>>> +fail_media_clean:
>>>>> +media_entity_cleanup(&vdev->entity);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void
>>>>> +mtk_cam_video_unregister_device(struct mtk_cam_video_device *node)
>>>>> +{
>>>>> +video_unregister_device(&node->vdev);
>>>>> +vb2_queue_release(&node->vbq);
>>>>> +media_entity_cleanup(&node->vdev.entity);
>>>>> +mutex_destroy(&node->vdev_lock);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int i, ret;
>>>>> +
>>>>> +/* Set up media device & pads */
>>>>> +ret = mtk_cam_media_register(cam, &cam->media_dev);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +dev_info(dev, "Registered media%d\n", cam->media_dev.devnode->minor);
>>>>> +
>>>>> +/* Set up v4l2 device */
>>>>> +cam->v4l2_dev.mdev = &cam->media_dev;
>>>>> +ret = v4l2_device_register(dev, &cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register V4L2 device:%d\n", ret);
>>>>> +goto fail_media_unreg;
>>>>> +}
>>>>> +dev_info(dev, "Registered %s\n", cam->v4l2_dev.name);
>>>>> +
>>>>> +/* Initialize subdev */
>>>>> +v4l2_subdev_init(&cam->subdev, &mtk_cam_subdev_ops);
>>>>> +cam->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>>>>> +cam->subdev.entity.ops = &mtk_cam_media_entity_ops;
>>>>> +cam->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>>>>> +V4L2_SUBDEV_FL_HAS_EVENTS;
>>>>> +snprintf(cam->subdev.name, sizeof(cam->subdev.name),
>>>>> + "%s", dev_driver_string(dev));
>>>>> +v4l2_set_subdevdata(&cam->subdev, cam);
>>>>> +
>>>>> +ret = v4l2_device_register_subdev(&cam->v4l2_dev, &cam->subdev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev:%d\n", ret);
>>>>> +goto fail_clean_media_entiy;
>>>>> +}
>>>>> +dev_dbg(dev, "registered %s\n", cam->subdev.name);
>>>>> +
>>>>> +/* Create video nodes and links */
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++) {
>>>>> +struct mtk_cam_video_device *node = &cam->vdev_nodes[i];
>>>>> +
>>>>> +node->id = node->desc.id;
>>>>> +ret = mtk_cam_video_register_device(cam, node);
>>>>> +if (ret)
>>>>> +goto fail_vdev_unreg;
>>>>> +}
>>>>> +vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_vdev_unreg:
>>>>> +for (i--; i >= 0; i--)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +fail_clean_media_entiy:
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +fail_media_unreg:
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_v4l2_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int i;
>>>>> +
>>>>> +for (i = 0; i < MTK_CAM_P1_TOTAL_NODES; i++)
>>>>> +mtk_cam_video_unregister_device(&cam->vdev_nodes[i]);
>>>>> +
>>>>> +vb2_dma_contig_clear_max_seg_size(cam->dev);
>>>>> +v4l2_device_unregister_subdev(&cam->subdev);
>>>>> +v4l2_device_unregister(&cam->v4l2_dev);
>>>>> +media_entity_cleanup(&cam->subdev.entity);
>>>>> +media_device_unregister(&cam->media_dev);
>>>>> +media_device_cleanup(&cam->media_dev);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_bound(struct v4l2_async_notifier *notifier,
>>>>> +      struct v4l2_subdev *sd,
>>>>> +      struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +if (!(sd->entity.function & MEDIA_ENT_F_VID_IF_BRIDGE)) {
>>>>> +dev_dbg(cam->dev, "no MEDIA_ENT_F_VID_IF_BRIDGE function\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +cam->seninf = sd;
>>>>> +dev_dbg(cam->dev, "%s is bound\n", sd->entity.name);
>>>>> +
>>>>> +return 0;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_dev_notifier_unbind(struct v4l2_async_notifier *notifier,
>>>>> +struct v4l2_subdev *sd,
>>>>> +struct v4l2_async_subdev *asd)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +
>>>>> +cam->seninf = NULL;
>>>>> +dev_dbg(cam->dev, "%s is unbound\n", sd->entity.name);
>>>>> +}
>>>>> +
>>>>> +static int mtk_cam_dev_notifier_complete(struct v4l2_async_notifier *notifier)
>>>>> +{
>>>>> +struct mtk_cam_dev *cam =
>>>>> +container_of(notifier, struct mtk_cam_dev, notifier);
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +if (!cam->seninf) {
>>>>> +dev_err(dev, "No seninf subdev\n");
>>>>> +return -ENODEV;
>>>>> +}
>>>>> +
>>>>> +ret = media_create_pad_link(&cam->seninf->entity, MTK_CAM_CIO_PAD_SRC,
>>>>> +    &cam->subdev.entity, MTK_CAM_CIO_PAD_SINK,
>>>>> +    MEDIA_LNK_FL_IMMUTABLE |
>>>>> +    MEDIA_LNK_FL_ENABLED);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to create pad link %s %s err:%d\n",
>>>>> +cam->seninf->entity.name, cam->subdev.entity.name,
>>>>> +ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +ret = v4l2_device_register_subdev_nodes(&cam->v4l2_dev);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to initialize subdev nodes:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_async_notifier_operations mtk_cam_v4l2_async_ops = {
>>>>> +.bound = mtk_cam_dev_notifier_bound,
>>>>> +.unbind = mtk_cam_dev_notifier_unbind,
>>>>> +.complete = mtk_cam_dev_notifier_complete,
>>>>> +};
>>>>> +
>>>>> +static int mtk_cam_v4l2_async_register(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +struct device *dev = cam->dev;
>>>>> +int ret;
>>>>> +
>>>>> +v4l2_async_notifier_init(&cam->notifier);
>>>>> +ret = v4l2_async_notifier_parse_fwnode_endpoints(dev,
>>>>> +&cam->notifier, sizeof(struct v4l2_async_subdev), NULL);
>>>>
>>>> It seems we shouldn't be using this function, please see comments at https://patchwork.kernel.org/patch/11066527/
>>>>
>>>> Regards,
>>>> Helen
>>>>
>>>
>>> Ok, we will investigate how to do.
>>>
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to parse fwnode endpoints:%d\n", ret);
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +cam->notifier.ops = &mtk_cam_v4l2_async_ops;
>>>>> +dev_dbg(dev, "mtk_cam v4l2_async_notifier_register\n");
>>>>> +ret = v4l2_async_notifier_register(&cam->v4l2_dev, &cam->notifier);
>>>>> +if (ret) {
>>>>> +dev_err(dev, "failed to register async notifier : %d\n", ret);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +static void mtk_cam_v4l2_async_unregister(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +v4l2_async_notifier_unregister(&cam->notifier);
>>>>> +v4l2_async_notifier_cleanup(&cam->notifier);
>>>>> +}
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_vcap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_framesizes = mtk_cam_vidioc_enum_framesizes,
>>>>> +.vidioc_enum_fmt_vid_cap = mtk_cam_vidioc_enum_fmt,
>>>>> +.vidioc_g_fmt_vid_cap_mplane = mtk_cam_vidioc_g_fmt,
>>>>> +.vidioc_s_fmt_vid_cap_mplane = mtk_cam_vidioc_s_fmt,
>>>>> +.vidioc_try_fmt_vid_cap_mplane = mtk_cam_vidioc_try_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>>>> +.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_cap_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_cap = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_cap = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_ioctl_ops mtk_cam_v4l2_meta_out_ioctl_ops = {
>>>>> +.vidioc_querycap = mtk_cam_vidioc_querycap,
>>>>> +.vidioc_enum_fmt_meta_out = mtk_cam_vidioc_meta_enum_fmt,
>>>>> +.vidioc_g_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_s_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_try_fmt_meta_out = mtk_cam_vidioc_g_meta_fmt,
>>>>> +.vidioc_reqbufs = vb2_ioctl_reqbufs,
>>>>> +.vidioc_create_bufs = vb2_ioctl_create_bufs,
>>>>> +.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>>>> +.vidioc_querybuf = vb2_ioctl_querybuf,
>>>>> +.vidioc_qbuf = vb2_ioctl_qbuf,
>>>>> +.vidioc_dqbuf = vb2_ioctl_dqbuf,
>>>>> +.vidioc_streamon = vb2_ioctl_streamon,
>>>>> +.vidioc_streamoff = vb2_ioctl_streamoff,
>>>>> +.vidioc_expbuf = vb2_ioctl_expbuf,
>>>>> +};> +
>>>>> +static const struct v4l2_format meta_fmts[] = {
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_PARAMS,
>>>>> +.buffersize = 512 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_3A,
>>>>> +.buffersize = 1200 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_AF,
>>>>> +.buffersize = 640 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LCS,
>>>>> +.buffersize = 288 * SZ_1K,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.meta = {
>>>>> +.dataformat = V4L2_META_FMT_MTISP_LMV,
>>>>> +.buffersize = 256,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format stream_out_fmts[] = {
>>>>> +/* This is a default image format */
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct v4l2_format bin_out_fmts[] = {
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SBGGR14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGBRG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SGRBG14F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB8F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB10F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB12F,
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.fmt.pix_mp = {
>>>>> +.width = IMG_MAX_WIDTH,
>>>>> +.height = IMG_MAX_HEIGHT,
>>>>> +.pixelformat = V4L2_PIX_FMT_MTISP_SRGGB14F,
>>>>> +},
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc output_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_IN_0,
>>>>> +.name = "meta input",
>>>>> +.cap = V4L2_CAP_META_OUTPUT,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_OUTPUT,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = true,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 0,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_out_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +static const struct
>>>>> +mtk_cam_dev_node_desc capture_queues[] = {
>>>>> +{
>>>>> +.id = MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +.name = "main stream",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_IMGO,
>>>>> +.fmts = stream_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(stream_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +.name = "packed out",
>>>>> +.cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE,
>>>>> +.buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>>>> +.link_flags = 0,
>>>>> +.image = true,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_RRZO,
>>>>> +.fmts = bin_out_fmts,
>>>>> +.num_fmts = ARRAY_SIZE(bin_out_fmts),
>>>>> +.default_fmt_idx = 0,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_vcap_ioctl_ops,
>>>>> +.frmsizes = &(struct v4l2_frmsizeenum) {
>>>>> +.index = 0,
>>>>> +.type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
>>>>> +.stepwise = {
>>>>> +.max_width = IMG_MAX_WIDTH,
>>>>> +.min_width = IMG_MIN_WIDTH,
>>>>> +.max_height = IMG_MAX_HEIGHT,
>>>>> +.min_height = IMG_MIN_HEIGHT,
>>>>> +.step_height = 1,
>>>>> +.step_width = 1,
>>>>> +},
>>>>> +},
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_0,
>>>>> +.name = "partial meta 0",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AAO | R_FLKO | R_PSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 1,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_1,
>>>>> +.name = "partial meta 1",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_AFO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 2,
>>>>> +.max_buf_count = 5,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_2,
>>>>> +.name = "partial meta 2",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LCSO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 3,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +{
>>>>> +.id = MTK_CAM_P1_META_OUT_3,
>>>>> +.name = "partial meta 3",
>>>>> +.cap = V4L2_CAP_META_CAPTURE,
>>>>> +.buf_type = V4L2_BUF_TYPE_META_CAPTURE,
>>>>> +.link_flags = 0,
>>>>> +.image = false,
>>>>> +.smem_alloc = false,
>>>>> +.dma_port = R_LMVO,
>>>>> +.fmts = meta_fmts,
>>>>> +.default_fmt_idx = 4,
>>>>> +.max_buf_count = 10,
>>>>> +.ioctl_ops = &mtk_cam_v4l2_meta_cap_ioctl_ops,
>>>>> +},
>>>>> +};
>>>>> +
>>>>> +/* The helper to configure the device context */
>>>>> +static void mtk_cam_dev_queue_setup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +unsigned int node_idx;
>>>>> +int i;
>>>>> +
>>>>> +node_idx = 0;
>>>>> +/* Setup the output queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(output_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = output_queues[i];
>>>>> +
>>>>> +/* Setup the capture queue */
>>>>> +for (i = 0; i < ARRAY_SIZE(capture_queues); i++)
>>>>> +cam->vdev_nodes[node_idx++].desc = capture_queues[i];
>>>>> +}
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +int ret;
>>>>> +
>>>>> +cam->dev = &pdev->dev;
>>>>> +mtk_cam_dev_queue_setup(cam);
>>>>> +
>>>>> +spin_lock_init(&cam->pending_job_lock);
>>>>> +spin_lock_init(&cam->running_job_lock);
>>>>> +INIT_LIST_HEAD(&cam->pending_job_list);
>>>>> +INIT_LIST_HEAD(&cam->running_job_list);
>>>>> +mutex_init(&cam->op_lock);
>>>>> +
>>>>> +/* v4l2 sub-device registration */
>>>>> +ret = mtk_cam_v4l2_register(cam);
>>>>> +if (ret)
>>>>> +return ret;
>>>>> +
>>>>> +ret = mtk_cam_v4l2_async_register(cam);
>>>>> +if (ret)
>>>>> +goto fail_v4l2_unreg;
>>>>> +
>>>>> +return 0;
>>>>> +
>>>>> +fail_v4l2_unreg:
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +
>>>>> +return ret;
>>>>> +}
>>>>> +
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam)
>>>>> +{
>>>>> +mtk_cam_v4l2_async_unregister(cam);
>>>>> +mtk_cam_v4l2_unregister(cam);
>>>>> +mutex_destroy(&cam->op_lock);
>>>>> +}
>>>>> +
>>>>> diff --git a/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> new file mode 100644
>>>>> index 000000000000..0a340a1e65ea
>>>>> --- /dev/null
>>>>> +++ b/drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
>>>>> @@ -0,0 +1,244 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>>> +/*
>>>>> + * Copyright (c) 2019 MediaTek Inc.
>>>>> + */
>>>>> +
>>>>> +#ifndef __MTK_CAM_H__
>>>>> +#define __MTK_CAM_H__
>>>>> +
>>>>> +#include <linux/device.h>
>>>>> +#include <linux/types.h>
>>>>> +#include <linux/platform_device.h>
>>>>> +#include <linux/spinlock.h>
>>>>> +#include <linux/videodev2.h>
>>>>> +#include <media/v4l2-device.h>
>>>>> +#include <media/v4l2-ctrls.h>
>>>>> +#include <media/v4l2-subdev.h>
>>>>> +#include <media/videobuf2-core.h>
>>>>> +#include <media/videobuf2-v4l2.h>
>>>>> +
>>>>> +#include "mtk_cam-ipi.h"
>>>>> +
>>>>> +#define IMG_MAX_WIDTH5376
>>>>> +#define IMG_MAX_HEIGHT4032
>>>>> +#define IMG_MIN_WIDTH80
>>>>> +#define IMG_MIN_HEIGHT60
>>>>> +
>>>>> +/*
>>>>> + * ID enum value for struct mtk_cam_dev_node_desc:id
>>>>> + * or mtk_cam_video_device:id
>>>>> + */
>>>>> +enum  {
>>>>> +MTK_CAM_P1_META_IN_0 = 0,
>>>>> +MTK_CAM_P1_MAIN_STREAM_OUT,
>>>>> +MTK_CAM_P1_PACKED_BIN_OUT,
>>>>> +MTK_CAM_P1_META_OUT_0,
>>>>> +MTK_CAM_P1_META_OUT_1,
>>>>> +MTK_CAM_P1_META_OUT_2,
>>>>> +MTK_CAM_P1_META_OUT_3,
>>>>> +MTK_CAM_P1_TOTAL_NODES
>>>>> +};
>>>>> +
>>>>> +/* Supported image format list */
>>>>> +#define MTK_CAM_IMG_FMT_UNKNOWN0x0000
>>>>> +#define MTK_CAM_IMG_FMT_BAYER80x2200
>>>>> +#define MTK_CAM_IMG_FMT_BAYER100x2201
>>>>> +#define MTK_CAM_IMG_FMT_BAYER120x2202
>>>>> +#define MTK_CAM_IMG_FMT_BAYER140x2203
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER80x2204
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER100x2205
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER120x2206
>>>>> +#define MTK_CAM_IMG_FMT_FG_BAYER140x2207
>>>>> +
>>>>> +/* Supported bayer pixel order */
>>>>> +#define MTK_CAM_RAW_PXL_ID_B0
>>>>> +#define MTK_CAM_RAW_PXL_ID_GB1
>>>>> +#define MTK_CAM_RAW_PXL_ID_GR2
>>>>> +#define MTK_CAM_RAW_PXL_ID_R3
>>>>> +#define MTK_CAM_RAW_PXL_ID_UNKNOWN4
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_p1_frame_param - MTK ISP P1 driver frame parameters.
>>>>> + *
>>>>> + * @frame_seq_no: The frame sequence of frame in driver layer.
>>>>> + * @dma_bufs: The DMA buffer address information of enabled DMA nodes.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_p1_frame_param {
>>>>> +unsigned int frame_seq_no;
>>>>> +struct dma_buffer dma_bufs[MTK_CAM_P1_TOTAL_NODES];
>>>>> +} __packed;
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_request - MTK camera device request.
>>>>> + *
>>>>> + * @req: Embedded struct media request.
>>>>> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
>>>>> + * @frame_work: work queue entry for frame transmission to SCP.
>>>>> + * @list: List entry of the object for @struct mtk_cam_dev:
>>>>> + *        pending_job_list or running_job_list.
>>>>> + * @timestamp: Start of frame timestamp in ns
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_request {
>>>>> +struct media_request req;
>>>>> +struct mtk_p1_frame_param frame_params;
>>>>> +struct work_struct frame_work;
>>>>> +struct list_head list;
>>>>> +u64 timestamp;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_buffer - MTK camera device buffer.
>>>>> + *
>>>>> + * @vbb: Embedded struct vb2_v4l2_buffer.
>>>>> + * @list: List entry of the object for @struct mtk_cam_video_device:
>>>>> + *        buf_list.
>>>>> + * @daddr: The DMA address of this buffer.
>>>>> + * @scp_addr: The SCP address of this buffer which
>>>>> + *            is only supported for meta input node.
>>>>> + * @node_id: The vidoe node id which this buffer belongs to.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_buffer {
>>>>> +struct vb2_v4l2_buffer vbb;
>>>>> +struct list_head list;
>>>>> +/* Intenal part */
>>>>> +dma_addr_t daddr;
>>>>> +dma_addr_t scp_addr;
>>>>> +unsigned int node_id;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
>>>>> + *
>>>>> + * @id: id of the node
>>>>> + * @name: name of the node
>>>>> + * @cap: supported V4L2 capabilities
>>>>> + * @buf_type: supported V4L2 buffer type
>>>>> + * @dma_port: the dma ports associated to the node
>>>>> + * @link_flags: default media link flags
>>>>> + * @smem_alloc: using the smem_dev as alloc device or not
>>>>> + * @image: true for image node, false for meta node
>>>>> + * @num_fmts: the number of supported node formats
>>>>> + * @default_fmt_idx: default format of this node
>>>>> + * @max_buf_count: maximum VB2 buffer count
>>>>> + * @ioctl_ops:  mapped to v4l2_ioctl_ops
>>>>> + * @fmts: supported format
>>>>> + * @frmsizes: supported V4L2 frame size number
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev_node_desc {
>>>>> +u8 id;
>>>>> +const char *name;
>>>>> +u32 cap;
>>>>> +u32 buf_type;
>>>>> +u32 dma_port;
>>>>> +u32 link_flags;
>>>>> +u8 smem_alloc:1;
>>>>> +u8 image:1;
>>>>> +u8 num_fmts;
>>>>> +u8 default_fmt_idx;
>>>>> +u8 max_buf_count;
>>>>> +const struct v4l2_ioctl_ops *ioctl_ops;
>>>>> +const struct v4l2_format *fmts;
>>>>> +const struct v4l2_frmsizeenum *frmsizes;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_video_device - Mediatek video device structure
>>>>> + *
>>>>> + * @id: Id for index of mtk_cam_dev:vdev_nodes array
>>>>> + * @enabled: Indicate the video device is enabled or not
>>>>> + * @desc: The node description of video device
>>>>> + * @vdev_fmt: The V4L2 format of video device
>>>>> + * @vdev_pad: The media pad graph object of video device
>>>>> + * @vbq: A videobuf queue of video device
>>>>> + * @vdev: The video device instance
>>>>> + * @vdev_lock: Serializes vb2 queue and video device operations
>>>>> + * @buf_list: List for enqueue buffers
>>>>> + * @buf_list_lock: Lock used to protect buffer list.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_video_device {
>>>>> +unsigned int id;
>>>>> +unsigned int enabled;
>>>>> +struct mtk_cam_dev_node_desc desc;
>>>>> +struct v4l2_format vdev_fmt;
>>>>> +struct media_pad vdev_pad;
>>>>> +struct vb2_queue vbq;
>>>>> +struct video_device vdev;
>>>>> +/* Serializes vb2 queue and video device operations */
>>>>> +struct mutex vdev_lock;
>>>>> +struct list_head buf_list;
>>>>> +/* Lock used to protect buffer list */
>>>>> +spinlock_t buf_list_lock;
>>>>> +};
>>>>> +
>>>>> +/*
>>>>> + * struct mtk_cam_dev - Mediatek camera device structure.
>>>>> + *
>>>>> + * @dev: Pointer to device.
>>>>> + * @smem_pdev: Pointer to shared memory device.
>>>>> + * @pipeline: Media pipeline information.
>>>>> + * @media_dev: Media device instance.
>>>>> + * @subdev: The V4L2 sub-device instance.
>>>>> + * @v4l2_dev: The V4L2 device driver instance.
>>>>> + * @notifier: The v4l2_device notifier data.
>>>>> + * @subdev_pads: Pointer to the number of media pads of this sub-device.
>>>>> + * @vdev_nodes: The array list of mtk_cam_video_device nodes.
>>>>> + * @seninf: Pointer to the seninf sub-device.
>>>>> + * @sensor: Pointer to the active sensor V4L2 sub-device when streaming on.
>>>>> + * @streaming: Indicate the overall streaming status is on or off.
>>>>> + * @enabled_dmas: The enabled dma port information when streaming on.
>>>>> + * @enabled_count: Number of enabled video nodes
>>>>> + * @stream_count: Number of streaming video nodes
>>>>> + * @running_job_count: Nunber of running jobs in the HW driver.
>>>>> + * @pending_job_list: List to keep the media requests before en-queue into
>>>>> + *                    HW driver.
>>>>> + * @pending_job_lock: Protect the pending_job_list data & running_job_count.
>>>>> + * @running_job_list: List to keep the media requests after en-queue into
>>>>> + *                    HW driver.
>>>>> + * @running_job_lock: Protect the running_job_list data.
>>>>> + * @op_lock: Serializes driver's VB2 callback operations.
>>>>> + *
>>>>> + */
>>>>> +struct mtk_cam_dev {
>>>>> +struct device *dev;
>>>>> +struct device *smem_dev;
>>>>> +struct media_pipeline pipeline;
>>>>> +struct media_device media_dev;
>>>>> +struct v4l2_subdev subdev;
>>>>> +struct v4l2_device v4l2_dev;
>>>>> +struct v4l2_async_notifier notifier;
>>>>> +struct media_pad *subdev_pads;
>>>>> +struct mtk_cam_video_device vdev_nodes[MTK_CAM_P1_TOTAL_NODES];
>>>>> +struct v4l2_subdev *seninf;
>>>>> +struct v4l2_subdev *sensor;
>>>>> +unsigned int streaming;
>>>>> +unsigned int enabled_dmas;
>>>>> +unsigned int enabled_count;
>>>>> +unsigned int stream_count;
>>>>> +unsigned int running_job_count;
>>>>> +struct list_head pending_job_list;
>>>>> +/* Protect the pending_job_list data */
>>>>> +spinlock_t pending_job_lock;
>>>>> +struct list_head running_job_list;
>>>>> +/* Protect the running_job_list data & running_job_count */
>>>>> +spinlock_t running_job_lock;
>>>>> +/* Serializes driver's VB2 callback operations */
>>>>> +struct mutex op_lock;
>>>>> +};
>>>>> +
>>>>> +int mtk_cam_dev_init(struct platform_device *pdev,
>>>>> +     struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_cleanup(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_req_try_queue(struct mtk_cam_dev *cam_dev);
>>>>> +void mtk_cam_dev_dequeue_req_frame(struct mtk_cam_dev *cam_dev,
>>>>> +   unsigned int frame_seq_no);
>>>>> +void mtk_cam_dev_event_frame_sync(struct mtk_cam_dev *cam_dev,
>>>>> +  unsigned int frame_seq_no);
>>>>> +struct mtk_cam_dev_request *mtk_cam_dev_get_req(struct mtk_cam_dev *cam,
>>>>> +unsigned int frame_seq_no);
>>>>> +
>>>>> +#endif /* __MTK_CAM_H__ */
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-mediatek mailing list
>>>> Linux-mediatek@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
>>>
>>
>> Regards,
>> Helen
> 
> Thanks for your comment.
> 
> Best regards,
> 
> 
> Jungo
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
  2020-05-05 15:30               ` Helen Koike
  (?)
@ 2020-05-05 16:18                 ` Tomasz Figa
  -1 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2020-05-05 16:18 UTC (permalink / raw)
  To: Helen Koike
  Cc: Jungo Lin, Mauro Carvalho Chehab, Shik Chen, linux-devicetree,
	Sean Cheng (鄭昇弘),
	Suleiman Souhlal, Pi-Hsun Shih, srv_heupstream, Rob Herring,
	Ryan Yu (余孟修),
	Jerry-ch Chen, Frankie Chiu (邱文凱),
	Sj Huang, yuzhao, moderated list:ARM/Mediatek SoC support,
	zwisler, ddavenport, Frederic Chen (陳俊元),
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,,
	Linux Media Mailing List, Hans Verkuil, Laurent Pinchart,
	Matthias Brugger

On Tue, May 5, 2020 at 5:30 PM Helen Koike <helen.koike@collabora.com> wrote:
>
> Hi Jungo,
>
> On 5/4/20 9:40 AM, Jungo Lin wrote:
> >
> > Hi Helen;
> >
> > Sorry for late reply.
> > Please check my feedback & questions below.
> >
> > On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> >>
> >> Hi Jungo,
> >>
> >> On 4/10/20 7:32 AM, Jungo Lin wrote:
> >>> Hi Helen:
> >>>
> >>> Thanks for your comment.
> >>>
> >>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >>>> Hi Jungo,
> >>>>
> >>>> I was taking a look at this patchset, please see my comments below.
> >>>>
> >>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>>>> Hello,
> >>>>>
> >>>>> This patch series adding the driver for Pass 1 (P1) unit in
> >>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> >>>>> camera features of CrOS.
> >>>>>
> >>>>> Pass 1 unit processes image signal from sensor devices and accepts the
> >>>>> tuning parameters to adjust the image quality. It performs optical
> >>>>> black correction, defect pixel correction, W/IR imbalance correction
> >>>>> and lens shading correction for RAW processing.
> >>>>>
> >>>>> The driver is implemented with V4L2 and media controller framework so
> >>>>> we have the following entities to describe the ISP pass 1 path.
> >>>>>
> >>>>> (The current metadata interface used in meta input and partial meta
> >>>>> nodes is only a temporary solution to kick off the driver development
> >>>>> and is not ready to be reviewed yet.)
> >>>>>
> >>>>> 1. meta input (output video device): connect to ISP P1 sub device.
> >>>>> It accepts the tuning buffer from user.
> >>>>>
> >>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> >>>>> main stream and packed out video devices. When processing an image,
> >>>>> Pass 1 hardware supports multiple output images with different sizes
> >>>>> and formats so it needs two capture video devices ("main stream" and
> >>>>> "packed out") to return the image data to the user.
> >>>>>
> >>>>> 3. main stream (capture video device): return the processed image data
> >>>>> which is used in capture scenario.
> >>>>>
> >>>>> 4. packed out (capture video device): return the processed image data
> >>>>> which is used in preview scenario.
> >>>>>
> >>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> >>>>>
> >>>>> 6. partial meta 1 (capture video device): return the AF statistics.
> >>>>>
> >>>>> 7. partial meta 2 (capture video device): return the local contrast
> >>>>>    enhanced statistics.
> >>>>>
> >>>>> 8. partial meta 3 (capture video device): return the local motion
> >>>>>    vector statistics.
> >>>>>
> >>>>> The overall patches of the series is:
> >>>>>
> >>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> >>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> >>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> >>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >>>>>   Moreover, implement standard V4L2 video driver that utilizes
> >>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
> >>>>>   communication to compose ISP registers in the firmware.
> >>>>>
> >>>>> Here is ISP P1 media topology:
> >>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
> >>>>> which are implemented in below patch[1][2][3][4]:
> >>>>
> >>>> I would be nice if you could provide a branch with those applied.
> >>>>
> >>>
> >>> We apply those patches in the chromeos-4.19 to test.
> >>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> >>>
> >>>
> >>>>>
> >>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> >>>>> patch sets.
> >>>>>
> >>>>> /usr/bin/media-ctl -p -d /dev/media2
> >>>>>
> >>>>> Media controller API version 4.19.89
> >>>>>
> >>>>> Media device information
> >>>>> ------------------------
> >>>>> driver          mtk-cam-p1
> >>>>> model           mtk-cam-p1
> >>>>> serial
> >>>>> bus info        platform:1a000000.camisp
> >>>>> hw revision     0x0
> >>>>> driver version  4.19.89
> >>>>>
> >>>>> Device topology
> >>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
> >>>>
> >>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> >>>> Is this correct?
> >>>>
> >>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> >>>>
> >>>
> >>> Currently, we only support single-cam in this SoC with upstream driver.
> >>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> >>> we'd like to keep the naming to avoid confusion.
> >>
> >> I suppose this new Mediatek SoC would use this same driver?
> >> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
> >> naming will be different then, right? (I guess it's ok).
> >>
> >
> > Sorry, my last comment should be corrected.
> > The new Mediatek SoC with new ISP HW version will support multi-cam
> > capabilities with new upstream driver. Due to the new enrich function,
> > it may not reuse current driver.
>
> right, thanks for the clarification.
>
> >
> >>>
> >>>>>             type V4L2 subdev subtype Unknown flags 0
> >>>>>             device node name /dev/v4l-subdev0
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1 meta input":0 []
> >>>>
> >>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> >>>>
> >>>
> >>> Ok, we will revise our naming in next patch.
> >>>
> >>>>> pad1: Source
> >>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> >>>>
> >>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> >>>>
> >>>
> >>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> >>>
> >>>>> pad2: Source
> >>>>> -> "mtk-cam-p1 packed out":0 []
> >>>>
> >>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> >>>>
> >>>
> >>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> >>> padding in the memory. We may revise the naming in next patch.
> >>>
> >>>>> pad3: Source
> >>>>> -> "mtk-cam-p1 partial meta 0":0 []
> >>>>> pad4: Source
> >>>>> -> "mtk-cam-p1 partial meta 1":0 []
> >>>>> pad5: Source
> >>>>> -> "mtk-cam-p1 partial meta 2":0 []
> >>>>> pad6: Source
> >>>>> -> "mtk-cam-p1 partial meta 3":0 []
> >>>>
> >>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> >>>>
> >>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> >>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> >>>>
> >>>> I also would prefer to remove blank spaces.
> >>>>
> >>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> >>>>
> >>>
> >>> No, these links are optional to setup for userspace.
> >>
> >> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
> >> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
> >>
> >> I'm not sure if using the link state is the best option (please see my comment on 5/5).
> >> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
> >> interesting nodes at once.
> >>
> >>
> >> Regards,
> >> Helen
> >>
> >
> >
> > According to your suggestion, do you have sample code about "userspace
> > could do a request to enable the stream in all the interesting nodes at
> > once"? If this supports, it is helpful for us to simply our current
> > implementation.
>
> I was checking the request api docs [1] in more details and it seems this isn't possible, since
> "A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
> on MEDIA_IOC_REQUEST_ALLOC doc[2].
>
> [1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
> [2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc
>
> What bothers me with the current implementation is that users could correctly configure the topology, but when
> calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
> with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
> for userspace about what is wrong.

The problem isn't specific to Request API or ISPs. The same would
happen in case of memory to memory devices, which need streaming on
both OUTPUT and CAPTURE queues to be started.

>
> So I was trying to think about how this could be done differently.
> I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
> If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
> nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
> is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

I'd envision some API to tell the userspace which nodes are required
for streaming. This could be perhaps inferred from the media
controller topology. Then if there are some nodes not started yet,
poll()/DQBUF could return some error.

>
> Regards,
> Helen
>
>
> >
> >
> > Thanks,
> >
> >
> > Jungo
> >
> >
> >>> For naming part, we will align with driver source codes.
> >>>
> >>>>> pad7: Source
> >>>>> pad8: Source
> >>>>> pad9: Source
> >>>>> pad10: Source
> >>>>
> >>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> >>>>
> >>>
> >>> These pads are just reserved purpose.
> >>> We will plan to remove them in next patch.
> >>>
> >>> Thanks,
> >>>
> >>> Jungo
> >>>
> >>>> Regards,
> >>>> Helen
> >>>>
> >>>>> pad11: Sink
> >>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video2
> >>>>> pad0: Source
> >>>>> -> "mtk-cam-p1":0 []
> >>>>>
> >>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video3
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video4
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":2 []
> >>>>>
> >>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video5
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":3 []
> >>>>>
> >>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video6
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":4 []
> >>>>>
> >>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video7
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":5 []
> >>>>>
> >>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video8
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":6 []
> >>>>>
> >>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
> >>>>>              type V4L2 subdev subtype Unknown flags 0
> >>>>>              device node name /dev/v4l-subdev1
> >>>>> pad0: Sink
> >>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> >>>>> <- "ov8856 2-0010":0 [ENABLED]
> >>>>> pad1: Sink
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> <- "ov02a10 4-003d":0 []
> >>>>> pad2: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad3: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad4: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> >>>>> pad5: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad6: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad7: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad8: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad9: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad10: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad11: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>>
> >>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev2
> >>>>> pad0: Source
> >>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> >>>>> -> "1a040000.seninf":0 [ENABLED]
> >>>>>
> >>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
> >>>>>              type V4L2 subdev subtype Lens flags 0
> >>>>>              device node name /dev/v4l-subdev3
> >>>>>
> >>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev4
> >>>>> pad0: Source
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
> >>>>> -> "1a040000.seninf":1 []
> >>>>>
> >>>>> ===========
> >>>>> = history =
> >>>>> ===========
> >>>>>
> >>>>> version 6:
> >>>>>  - Add port node description in the dt-binding document and device tree
> >>>>>    by Tomasz Figa.
> >>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >>>>>    version.
> >>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >>>>>  - Correct auto suspend timer value for suspend/resume issue.
> >>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
> >>>>>    timeout event.
> >>>>>  - Fix KE due to no sen-inf sub-device.
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation.
> >>>>>  - Add rst documents for Mediatek meta formats.
> >>>>>  - New meta buffer structure design & re-factoring.
> >>>>>
> >>>>> version 5:
> >>>>>  - Fixed Rob's comment on dt-binding format
> >>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>    and new timestamp type in driver
> >>>>>  - Fix buffer en-queue timing issue in v4
> >>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 4:
> >>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >>>>>    patch[4]
> >>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >>>>>  - Extend Mediatek proprietary image formats to support bayer order
> >>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 3:
> >>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >>>>>    reserved memory region. (Rob Herring)
> >>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >>>>>  - Revise ISP Pass1 Kconfig
> >>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >>>>>  - Fix kernel warning messages when running v4l2_compliance test
> >>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
> >>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >>>>>    declaration (Hans Verkuil)
> >>>>>    Split GET_BIN_INFO control into two controls to get width & height
> >>>>>    in-dependently (Hans Verkuil)
> >>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Fix ISP de-initialize timing KE issue
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> >>>>>
> >>>>> Todo:
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>
> >>>>> version 2:
> >>>>>  - Add 3A enhancement feature which includes:
> >>>>>    Separates 3A pipeline out of frame basis to improve
> >>>>>    AE/AWB (exposure and white balance) performance.
> >>>>>    Add 2 SCP sub-commands for 3A meta buffers.
> >>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
> >>>>>    and co-processor.
> >>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >>>>>  - Revised document wording for dt-bindings documents & dts information.
> >>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
> >>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >>>>>    or add comments.
> >>>>>    Revised buffer size for LMVO & LCSO.
> >>>>>    Fix pixel format utility function.
> >>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >>>>>  - mtk_cam-v4l2-util.c
> >>>>>    Refactoring V4L2 async mechanism with seninf driver only
> >>>>>    Refactoring CIO (Connection IO) implementation with active sensor
> >>>>>    Revised stream on function for 3A enhancement feature
> >>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Add meta buffer index register definitions
> >>>>>    Add meta DMA configuration function.
> >>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >>>>>    Add isp_setup_scp_rproc function to get RPC handle
> >>>>>    Add mtk_cam_reserved_memory_init for shared memory management
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Add new meta strictures for 3A enhancement feature
> >>>>>    Add new IPI command utility function for 3A enhancement feature
> >>>>>    Enhance isp_composer_dma_sg_init function flow
> >>>>>    Shorten overall IPI command structure size
> >>>>>    Remove scp_state state checking
> >>>>>    Improve code readability
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >>>>>    memory region.
> >>>>>
> >>>>> TODOs:
> >>>>>  - 3A enhancement feature bug fixing
> >>>>>
> >>>>> version 1:
> >>>>>  - Revised driver sources based on Tomasz's comments including
> >>>>>    part1/2/3/4 in RFC V0 patch.
> >>>>>  - Remove DMA cache mechanism.
> >>>>>    Support two new video devices (LCSO/LMVO) for advance camera
> >>>>>    features.
> >>>>>  - Fixed v4l2-compliance test failure items.
> >>>>>  - Add private controls for Mediatek camera middle-ware.
> >>>>>  - Replace VPU driver's APIs with new SCP driver interface for
> >>>>>    co-processor communication.
> >>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >>>>>    commands RX handling.
> >>>>>  - Fix internal bugs.
> >>>>>
> >>>>> TODOs:
> >>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >>>>>    for shared memory management.
> >>>>>  - Revised file names.
> >>>>>  - Support non frame-sync AFO/AAO DMA buffers
> >>>>>
> >>>>> version 0:
> >>>>> - Initial submission
> >>>>>
> >>>>> ==================
> >>>>>  Dependent patch set
> >>>>> ==================
> >>>>>
> >>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
> >>>>> The patches are listed as following:
> >>>>>
> >>>>> [1]. media: support Mediatek sensor interface driver
> >>>>> https://patchwork.kernel.org/cover/11145845/
> >>>>>
> >>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
> >>>>> https://patchwork.kernel.org/cover/11220785/
> >>>>>
> >>>>> [3]. media: i2c: Add support for OV02A10 sensor
> >>>>> https://patchwork.kernel.org/cover/11284779/
> >>>>>
> >>>>> [4]. media: i2c: add support for DW9768 VCM driver
> >>>>> https://patchwork.kernel.org/cover/11132299/
> >>>>>
> >>>>> [5]. Add support for mt8183 SCP
> >>>>> https://patchwork.kernel.org/cover/11239065/
> >>>>>
> >>>>> [6]. MT8183 IOMMU SUPPORT
> >>>>> https://patchwork.kernel.org/cover/11112765/
> >>>>>
> >>>>> ==================
> >>>>>  Compliance test
> >>>>> ==================
> >>>>>
> >>>>> The v4l2-compliance is built with the below lastest patch.
> >>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> >>>>>
> >>>>> Note 1.
> >>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> >>>>>
> >>>>> Note 2.
> >>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> >>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> >>>>>
> >>>>> Note 3.
> >>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> >>>>>
> >>>>> /usr/bin/v4l2-compliance -m /dev/media2
> >>>>>
> >>>>> v4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media1 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video25:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.67
> >>>>> Capabilities     : 0x8c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video25 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composiv4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media2 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video2:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x8c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video2 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video3:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000016
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000014 (20)
> >>>>> Name             : mtk-cam-p1 main stream
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000015   : 0: Sink
> >>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video3 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video4:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300001c
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000001a (26)
> >>>>> Name             : mtk-cam-p1 packed out
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100001b   : 0: Sink
> >>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video4 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video5:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000022
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000020 (32)
> >>>>> Name             : mtk-cam-p1 partial meta 0
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000021   : 0: Sink
> >>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video5 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video6:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000028
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000026 (38)
> >>>>> Name             : mtk-cam-p1 partial meta 1
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000027   : 0: Sink
> >>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video6 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video7:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300002e
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000002c (44)
> >>>>> Name             : mtk-cam-p1 partial meta 2
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100002d   : 0: Sink
> >>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video7 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video8:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000034
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000032 (50)
> >>>>> Name             : mtk-cam-p1 partial meta 3
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000033   : 0: Sink
> >>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video8 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000050
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000001 (1)
> >>>>> Name             : mtk-cam-p1
> >>>>> Function         : Video Pixel Formatter
> >>>>> Pad 0x01000002   : 0: Sink
> >>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> >>>>> Pad 0x01000003   : 1: Source
> >>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> >>>>> Pad 0x01000004   : 2: Source
> >>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> >>>>> Pad 0x01000005   : 3: Source
> >>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> >>>>> Pad 0x01000006   : 4: Source
> >>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> >>>>> Pad 0x01000007   : 5: Source
> >>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> >>>>> Pad 0x01000008   : 6: Source
> >>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> >>>>> Pad 0x01000009   : 7: Source
> >>>>> Pad 0x0100000a   : 8: Source
> >>>>> Pad 0x0100000b   : 9: Source
> >>>>> Pad 0x0100000c   : 10: Source
> >>>>> Pad 0x0100000d   : 11: Sink
> >>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev0 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000052
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000038 (56)
> >>>>> Name             : 1a040000.seninf
> >>>>> Function         : Video Interface Bridge
> >>>>> Pad 0x01000039   : 0: Sink
> >>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> >>>>> Pad 0x0100003a   : 1: Sink
> >>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> >>>>> Pad 0x0100003b   : 2: Sink
> >>>>> Pad 0x0100003c   : 3: Sink
> >>>>> Pad 0x0100003d   : 4: Source
> >>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>> Pad 0x0100003e   : 5: Source
> >>>>> Pad 0x0100003f   : 6: Source
> >>>>> Pad 0x01000040   : 7: Source
> >>>>> Pad 0x01000041   : 8: Source
> >>>>> Pad 0x01000042   : 9: Source
> >>>>> Pad 0x01000043   : 10: Source
> >>>>> Pad 0x01000044   : 11: Source
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev1 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000054
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000045 (69)
> >>>>> Name             : ov8856 2-0010
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x01000046   : 0: Source
> >>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev2 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> >>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 11 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000056
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000049 (73)
> >>>>> Name             : dw9768 2-000c
> >>>>> Function         : Lens Controller
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev3 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000058
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x0000004a (74)
> >>>>> Name             : ov02a10 4-003d
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x0100004b   : 0: Source
> >>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev4 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> >>>>> test VIDIOC_G/S_CTRL: FAIL
> >>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 10 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> >>>>>
> >>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> >>>>>
> >>>>>
> >>>>> Jungo Lin (5):
> >>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >>>>>   media: videodev2.h: Add new boottime timestamp type
> >>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
> >>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> >>>>>
> >>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
> >>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >>>>>  drivers/media/platform/Kconfig                |    1 +
> >>>>>  drivers/media/platform/Makefile               |    1 +
> >>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >>>>>  include/uapi/linux/videodev2.h                |   41 +
> >>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
> >>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> Linux-mediatek mailing list
> >>>> Linux-mediatek@lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> >

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-05-05 16:18                 ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2020-05-05 16:18 UTC (permalink / raw)
  To: Helen Koike
  Cc: Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Laurent Pinchart, Rob Herring, Suleiman Souhlal, Jerry-ch Chen,
	Jungo Lin, Frederic Chen (陳俊元),
	Linux Media Mailing List, linux-devicetree, ddavenport, Sj Huang,
	yuzhao, moderated list:ARM/Mediatek SoC support, Pi-Hsun Shih,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Sean Cheng (鄭昇弘),
	srv_heupstream, Shik Chen, zwisler, Hans Verkuil

On Tue, May 5, 2020 at 5:30 PM Helen Koike <helen.koike@collabora.com> wrote:
>
> Hi Jungo,
>
> On 5/4/20 9:40 AM, Jungo Lin wrote:
> >
> > Hi Helen;
> >
> > Sorry for late reply.
> > Please check my feedback & questions below.
> >
> > On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> >>
> >> Hi Jungo,
> >>
> >> On 4/10/20 7:32 AM, Jungo Lin wrote:
> >>> Hi Helen:
> >>>
> >>> Thanks for your comment.
> >>>
> >>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >>>> Hi Jungo,
> >>>>
> >>>> I was taking a look at this patchset, please see my comments below.
> >>>>
> >>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>>>> Hello,
> >>>>>
> >>>>> This patch series adding the driver for Pass 1 (P1) unit in
> >>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> >>>>> camera features of CrOS.
> >>>>>
> >>>>> Pass 1 unit processes image signal from sensor devices and accepts the
> >>>>> tuning parameters to adjust the image quality. It performs optical
> >>>>> black correction, defect pixel correction, W/IR imbalance correction
> >>>>> and lens shading correction for RAW processing.
> >>>>>
> >>>>> The driver is implemented with V4L2 and media controller framework so
> >>>>> we have the following entities to describe the ISP pass 1 path.
> >>>>>
> >>>>> (The current metadata interface used in meta input and partial meta
> >>>>> nodes is only a temporary solution to kick off the driver development
> >>>>> and is not ready to be reviewed yet.)
> >>>>>
> >>>>> 1. meta input (output video device): connect to ISP P1 sub device.
> >>>>> It accepts the tuning buffer from user.
> >>>>>
> >>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> >>>>> main stream and packed out video devices. When processing an image,
> >>>>> Pass 1 hardware supports multiple output images with different sizes
> >>>>> and formats so it needs two capture video devices ("main stream" and
> >>>>> "packed out") to return the image data to the user.
> >>>>>
> >>>>> 3. main stream (capture video device): return the processed image data
> >>>>> which is used in capture scenario.
> >>>>>
> >>>>> 4. packed out (capture video device): return the processed image data
> >>>>> which is used in preview scenario.
> >>>>>
> >>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> >>>>>
> >>>>> 6. partial meta 1 (capture video device): return the AF statistics.
> >>>>>
> >>>>> 7. partial meta 2 (capture video device): return the local contrast
> >>>>>    enhanced statistics.
> >>>>>
> >>>>> 8. partial meta 3 (capture video device): return the local motion
> >>>>>    vector statistics.
> >>>>>
> >>>>> The overall patches of the series is:
> >>>>>
> >>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> >>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> >>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> >>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >>>>>   Moreover, implement standard V4L2 video driver that utilizes
> >>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
> >>>>>   communication to compose ISP registers in the firmware.
> >>>>>
> >>>>> Here is ISP P1 media topology:
> >>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
> >>>>> which are implemented in below patch[1][2][3][4]:
> >>>>
> >>>> I would be nice if you could provide a branch with those applied.
> >>>>
> >>>
> >>> We apply those patches in the chromeos-4.19 to test.
> >>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> >>>
> >>>
> >>>>>
> >>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> >>>>> patch sets.
> >>>>>
> >>>>> /usr/bin/media-ctl -p -d /dev/media2
> >>>>>
> >>>>> Media controller API version 4.19.89
> >>>>>
> >>>>> Media device information
> >>>>> ------------------------
> >>>>> driver          mtk-cam-p1
> >>>>> model           mtk-cam-p1
> >>>>> serial
> >>>>> bus info        platform:1a000000.camisp
> >>>>> hw revision     0x0
> >>>>> driver version  4.19.89
> >>>>>
> >>>>> Device topology
> >>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
> >>>>
> >>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> >>>> Is this correct?
> >>>>
> >>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> >>>>
> >>>
> >>> Currently, we only support single-cam in this SoC with upstream driver.
> >>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> >>> we'd like to keep the naming to avoid confusion.
> >>
> >> I suppose this new Mediatek SoC would use this same driver?
> >> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
> >> naming will be different then, right? (I guess it's ok).
> >>
> >
> > Sorry, my last comment should be corrected.
> > The new Mediatek SoC with new ISP HW version will support multi-cam
> > capabilities with new upstream driver. Due to the new enrich function,
> > it may not reuse current driver.
>
> right, thanks for the clarification.
>
> >
> >>>
> >>>>>             type V4L2 subdev subtype Unknown flags 0
> >>>>>             device node name /dev/v4l-subdev0
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1 meta input":0 []
> >>>>
> >>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> >>>>
> >>>
> >>> Ok, we will revise our naming in next patch.
> >>>
> >>>>> pad1: Source
> >>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> >>>>
> >>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> >>>>
> >>>
> >>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> >>>
> >>>>> pad2: Source
> >>>>> -> "mtk-cam-p1 packed out":0 []
> >>>>
> >>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> >>>>
> >>>
> >>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> >>> padding in the memory. We may revise the naming in next patch.
> >>>
> >>>>> pad3: Source
> >>>>> -> "mtk-cam-p1 partial meta 0":0 []
> >>>>> pad4: Source
> >>>>> -> "mtk-cam-p1 partial meta 1":0 []
> >>>>> pad5: Source
> >>>>> -> "mtk-cam-p1 partial meta 2":0 []
> >>>>> pad6: Source
> >>>>> -> "mtk-cam-p1 partial meta 3":0 []
> >>>>
> >>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> >>>>
> >>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> >>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> >>>>
> >>>> I also would prefer to remove blank spaces.
> >>>>
> >>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> >>>>
> >>>
> >>> No, these links are optional to setup for userspace.
> >>
> >> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
> >> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
> >>
> >> I'm not sure if using the link state is the best option (please see my comment on 5/5).
> >> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
> >> interesting nodes at once.
> >>
> >>
> >> Regards,
> >> Helen
> >>
> >
> >
> > According to your suggestion, do you have sample code about "userspace
> > could do a request to enable the stream in all the interesting nodes at
> > once"? If this supports, it is helpful for us to simply our current
> > implementation.
>
> I was checking the request api docs [1] in more details and it seems this isn't possible, since
> "A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
> on MEDIA_IOC_REQUEST_ALLOC doc[2].
>
> [1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
> [2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc
>
> What bothers me with the current implementation is that users could correctly configure the topology, but when
> calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
> with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
> for userspace about what is wrong.

The problem isn't specific to Request API or ISPs. The same would
happen in case of memory to memory devices, which need streaming on
both OUTPUT and CAPTURE queues to be started.

>
> So I was trying to think about how this could be done differently.
> I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
> If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
> nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
> is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

I'd envision some API to tell the userspace which nodes are required
for streaming. This could be perhaps inferred from the media
controller topology. Then if there are some nodes not started yet,
poll()/DQBUF could return some error.

>
> Regards,
> Helen
>
>
> >
> >
> > Thanks,
> >
> >
> > Jungo
> >
> >
> >>> For naming part, we will align with driver source codes.
> >>>
> >>>>> pad7: Source
> >>>>> pad8: Source
> >>>>> pad9: Source
> >>>>> pad10: Source
> >>>>
> >>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> >>>>
> >>>
> >>> These pads are just reserved purpose.
> >>> We will plan to remove them in next patch.
> >>>
> >>> Thanks,
> >>>
> >>> Jungo
> >>>
> >>>> Regards,
> >>>> Helen
> >>>>
> >>>>> pad11: Sink
> >>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video2
> >>>>> pad0: Source
> >>>>> -> "mtk-cam-p1":0 []
> >>>>>
> >>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video3
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video4
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":2 []
> >>>>>
> >>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video5
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":3 []
> >>>>>
> >>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video6
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":4 []
> >>>>>
> >>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video7
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":5 []
> >>>>>
> >>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video8
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":6 []
> >>>>>
> >>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
> >>>>>              type V4L2 subdev subtype Unknown flags 0
> >>>>>              device node name /dev/v4l-subdev1
> >>>>> pad0: Sink
> >>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> >>>>> <- "ov8856 2-0010":0 [ENABLED]
> >>>>> pad1: Sink
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> <- "ov02a10 4-003d":0 []
> >>>>> pad2: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad3: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad4: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> >>>>> pad5: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad6: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad7: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad8: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad9: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad10: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad11: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>>
> >>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev2
> >>>>> pad0: Source
> >>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> >>>>> -> "1a040000.seninf":0 [ENABLED]
> >>>>>
> >>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
> >>>>>              type V4L2 subdev subtype Lens flags 0
> >>>>>              device node name /dev/v4l-subdev3
> >>>>>
> >>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev4
> >>>>> pad0: Source
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
> >>>>> -> "1a040000.seninf":1 []
> >>>>>
> >>>>> ===========
> >>>>> = history =
> >>>>> ===========
> >>>>>
> >>>>> version 6:
> >>>>>  - Add port node description in the dt-binding document and device tree
> >>>>>    by Tomasz Figa.
> >>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >>>>>    version.
> >>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >>>>>  - Correct auto suspend timer value for suspend/resume issue.
> >>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
> >>>>>    timeout event.
> >>>>>  - Fix KE due to no sen-inf sub-device.
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation.
> >>>>>  - Add rst documents for Mediatek meta formats.
> >>>>>  - New meta buffer structure design & re-factoring.
> >>>>>
> >>>>> version 5:
> >>>>>  - Fixed Rob's comment on dt-binding format
> >>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>    and new timestamp type in driver
> >>>>>  - Fix buffer en-queue timing issue in v4
> >>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 4:
> >>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >>>>>    patch[4]
> >>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >>>>>  - Extend Mediatek proprietary image formats to support bayer order
> >>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 3:
> >>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >>>>>    reserved memory region. (Rob Herring)
> >>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >>>>>  - Revise ISP Pass1 Kconfig
> >>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >>>>>  - Fix kernel warning messages when running v4l2_compliance test
> >>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
> >>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >>>>>    declaration (Hans Verkuil)
> >>>>>    Split GET_BIN_INFO control into two controls to get width & height
> >>>>>    in-dependently (Hans Verkuil)
> >>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Fix ISP de-initialize timing KE issue
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> >>>>>
> >>>>> Todo:
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>
> >>>>> version 2:
> >>>>>  - Add 3A enhancement feature which includes:
> >>>>>    Separates 3A pipeline out of frame basis to improve
> >>>>>    AE/AWB (exposure and white balance) performance.
> >>>>>    Add 2 SCP sub-commands for 3A meta buffers.
> >>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
> >>>>>    and co-processor.
> >>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >>>>>  - Revised document wording for dt-bindings documents & dts information.
> >>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
> >>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >>>>>    or add comments.
> >>>>>    Revised buffer size for LMVO & LCSO.
> >>>>>    Fix pixel format utility function.
> >>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >>>>>  - mtk_cam-v4l2-util.c
> >>>>>    Refactoring V4L2 async mechanism with seninf driver only
> >>>>>    Refactoring CIO (Connection IO) implementation with active sensor
> >>>>>    Revised stream on function for 3A enhancement feature
> >>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Add meta buffer index register definitions
> >>>>>    Add meta DMA configuration function.
> >>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >>>>>    Add isp_setup_scp_rproc function to get RPC handle
> >>>>>    Add mtk_cam_reserved_memory_init for shared memory management
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Add new meta strictures for 3A enhancement feature
> >>>>>    Add new IPI command utility function for 3A enhancement feature
> >>>>>    Enhance isp_composer_dma_sg_init function flow
> >>>>>    Shorten overall IPI command structure size
> >>>>>    Remove scp_state state checking
> >>>>>    Improve code readability
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >>>>>    memory region.
> >>>>>
> >>>>> TODOs:
> >>>>>  - 3A enhancement feature bug fixing
> >>>>>
> >>>>> version 1:
> >>>>>  - Revised driver sources based on Tomasz's comments including
> >>>>>    part1/2/3/4 in RFC V0 patch.
> >>>>>  - Remove DMA cache mechanism.
> >>>>>    Support two new video devices (LCSO/LMVO) for advance camera
> >>>>>    features.
> >>>>>  - Fixed v4l2-compliance test failure items.
> >>>>>  - Add private controls for Mediatek camera middle-ware.
> >>>>>  - Replace VPU driver's APIs with new SCP driver interface for
> >>>>>    co-processor communication.
> >>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >>>>>    commands RX handling.
> >>>>>  - Fix internal bugs.
> >>>>>
> >>>>> TODOs:
> >>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >>>>>    for shared memory management.
> >>>>>  - Revised file names.
> >>>>>  - Support non frame-sync AFO/AAO DMA buffers
> >>>>>
> >>>>> version 0:
> >>>>> - Initial submission
> >>>>>
> >>>>> ==================
> >>>>>  Dependent patch set
> >>>>> ==================
> >>>>>
> >>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
> >>>>> The patches are listed as following:
> >>>>>
> >>>>> [1]. media: support Mediatek sensor interface driver
> >>>>> https://patchwork.kernel.org/cover/11145845/
> >>>>>
> >>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
> >>>>> https://patchwork.kernel.org/cover/11220785/
> >>>>>
> >>>>> [3]. media: i2c: Add support for OV02A10 sensor
> >>>>> https://patchwork.kernel.org/cover/11284779/
> >>>>>
> >>>>> [4]. media: i2c: add support for DW9768 VCM driver
> >>>>> https://patchwork.kernel.org/cover/11132299/
> >>>>>
> >>>>> [5]. Add support for mt8183 SCP
> >>>>> https://patchwork.kernel.org/cover/11239065/
> >>>>>
> >>>>> [6]. MT8183 IOMMU SUPPORT
> >>>>> https://patchwork.kernel.org/cover/11112765/
> >>>>>
> >>>>> ==================
> >>>>>  Compliance test
> >>>>> ==================
> >>>>>
> >>>>> The v4l2-compliance is built with the below lastest patch.
> >>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> >>>>>
> >>>>> Note 1.
> >>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> >>>>>
> >>>>> Note 2.
> >>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> >>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> >>>>>
> >>>>> Note 3.
> >>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> >>>>>
> >>>>> /usr/bin/v4l2-compliance -m /dev/media2
> >>>>>
> >>>>> v4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media1 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video25:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.67
> >>>>> Capabilities     : 0x8c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video25 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composiv4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media2 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video2:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x8c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video2 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video3:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000016
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000014 (20)
> >>>>> Name             : mtk-cam-p1 main stream
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000015   : 0: Sink
> >>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video3 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video4:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300001c
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000001a (26)
> >>>>> Name             : mtk-cam-p1 packed out
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100001b   : 0: Sink
> >>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video4 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video5:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000022
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000020 (32)
> >>>>> Name             : mtk-cam-p1 partial meta 0
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000021   : 0: Sink
> >>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video5 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video6:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000028
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000026 (38)
> >>>>> Name             : mtk-cam-p1 partial meta 1
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000027   : 0: Sink
> >>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video6 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video7:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300002e
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000002c (44)
> >>>>> Name             : mtk-cam-p1 partial meta 2
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100002d   : 0: Sink
> >>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video7 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video8:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000034
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000032 (50)
> >>>>> Name             : mtk-cam-p1 partial meta 3
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000033   : 0: Sink
> >>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video8 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000050
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000001 (1)
> >>>>> Name             : mtk-cam-p1
> >>>>> Function         : Video Pixel Formatter
> >>>>> Pad 0x01000002   : 0: Sink
> >>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> >>>>> Pad 0x01000003   : 1: Source
> >>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> >>>>> Pad 0x01000004   : 2: Source
> >>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> >>>>> Pad 0x01000005   : 3: Source
> >>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> >>>>> Pad 0x01000006   : 4: Source
> >>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> >>>>> Pad 0x01000007   : 5: Source
> >>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> >>>>> Pad 0x01000008   : 6: Source
> >>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> >>>>> Pad 0x01000009   : 7: Source
> >>>>> Pad 0x0100000a   : 8: Source
> >>>>> Pad 0x0100000b   : 9: Source
> >>>>> Pad 0x0100000c   : 10: Source
> >>>>> Pad 0x0100000d   : 11: Sink
> >>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev0 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000052
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000038 (56)
> >>>>> Name             : 1a040000.seninf
> >>>>> Function         : Video Interface Bridge
> >>>>> Pad 0x01000039   : 0: Sink
> >>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> >>>>> Pad 0x0100003a   : 1: Sink
> >>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> >>>>> Pad 0x0100003b   : 2: Sink
> >>>>> Pad 0x0100003c   : 3: Sink
> >>>>> Pad 0x0100003d   : 4: Source
> >>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>> Pad 0x0100003e   : 5: Source
> >>>>> Pad 0x0100003f   : 6: Source
> >>>>> Pad 0x01000040   : 7: Source
> >>>>> Pad 0x01000041   : 8: Source
> >>>>> Pad 0x01000042   : 9: Source
> >>>>> Pad 0x01000043   : 10: Source
> >>>>> Pad 0x01000044   : 11: Source
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev1 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000054
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000045 (69)
> >>>>> Name             : ov8856 2-0010
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x01000046   : 0: Source
> >>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev2 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> >>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 11 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000056
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000049 (73)
> >>>>> Name             : dw9768 2-000c
> >>>>> Function         : Lens Controller
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev3 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000058
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x0000004a (74)
> >>>>> Name             : ov02a10 4-003d
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x0100004b   : 0: Source
> >>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev4 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> >>>>> test VIDIOC_G/S_CTRL: FAIL
> >>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 10 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> >>>>>
> >>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> >>>>>
> >>>>>
> >>>>> Jungo Lin (5):
> >>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >>>>>   media: videodev2.h: Add new boottime timestamp type
> >>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
> >>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> >>>>>
> >>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
> >>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >>>>>  drivers/media/platform/Kconfig                |    1 +
> >>>>>  drivers/media/platform/Makefile               |    1 +
> >>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >>>>>  include/uapi/linux/videodev2.h                |   41 +
> >>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
> >>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> Linux-mediatek mailing list
> >>>> Linux-mediatek@lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> >

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

^ permalink raw reply	[flat|nested] 388+ messages in thread

* Re: [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver
@ 2020-05-05 16:18                 ` Tomasz Figa
  0 siblings, 0 replies; 388+ messages in thread
From: Tomasz Figa @ 2020-05-05 16:18 UTC (permalink / raw)
  To: Helen Koike
  Cc: Ryan Yu (余孟修),
	Frankie Chiu (邱文凱),
	Laurent Pinchart, Rob Herring, Suleiman Souhlal, Jerry-ch Chen,
	Jungo Lin, Frederic Chen (陳俊元),
	Linux Media Mailing List, linux-devicetree, ddavenport, Sj Huang,
	yuzhao, moderated list:ARM/Mediatek SoC support, Pi-Hsun Shih,
	Matthias Brugger, Mauro Carvalho Chehab,
	list@263.net:IOMMU DRIVERS
	<iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>, ,
	Sean Cheng (鄭昇弘),
	srv_heupstream, Shik Chen, zwisler, Hans Verkuil

On Tue, May 5, 2020 at 5:30 PM Helen Koike <helen.koike@collabora.com> wrote:
>
> Hi Jungo,
>
> On 5/4/20 9:40 AM, Jungo Lin wrote:
> >
> > Hi Helen;
> >
> > Sorry for late reply.
> > Please check my feedback & questions below.
> >
> > On Tue, 2020-04-14 at 09:25 -0300, Helen Koike wrote:
> >>
> >> Hi Jungo,
> >>
> >> On 4/10/20 7:32 AM, Jungo Lin wrote:
> >>> Hi Helen:
> >>>
> >>> Thanks for your comment.
> >>>
> >>> On Tue, 2020-03-31 at 12:34 -0300, Helen Koike wrote:
> >>>> Hi Jungo,
> >>>>
> >>>> I was taking a look at this patchset, please see my comments below.
> >>>>
> >>>> On 12/19/19 3:49 AM, Jungo Lin wrote:
> >>>>> Hello,
> >>>>>
> >>>>> This patch series adding the driver for Pass 1 (P1) unit in
> >>>>> Mediatek's camera ISP system on mt8183 SoC, which will be used in
> >>>>> camera features of CrOS.
> >>>>>
> >>>>> Pass 1 unit processes image signal from sensor devices and accepts the
> >>>>> tuning parameters to adjust the image quality. It performs optical
> >>>>> black correction, defect pixel correction, W/IR imbalance correction
> >>>>> and lens shading correction for RAW processing.
> >>>>>
> >>>>> The driver is implemented with V4L2 and media controller framework so
> >>>>> we have the following entities to describe the ISP pass 1 path.
> >>>>>
> >>>>> (The current metadata interface used in meta input and partial meta
> >>>>> nodes is only a temporary solution to kick off the driver development
> >>>>> and is not ready to be reviewed yet.)
> >>>>>
> >>>>> 1. meta input (output video device): connect to ISP P1 sub device.
> >>>>> It accepts the tuning buffer from user.
> >>>>>
> >>>>> 2. ISP P1 (sub device): connect to partial meta 0/1/2/3,
> >>>>> main stream and packed out video devices. When processing an image,
> >>>>> Pass 1 hardware supports multiple output images with different sizes
> >>>>> and formats so it needs two capture video devices ("main stream" and
> >>>>> "packed out") to return the image data to the user.
> >>>>>
> >>>>> 3. main stream (capture video device): return the processed image data
> >>>>> which is used in capture scenario.
> >>>>>
> >>>>> 4. packed out (capture video device): return the processed image data
> >>>>> which is used in preview scenario.
> >>>>>
> >>>>> 5. partial meta 0 (capture video device): return the AE/AWB statistics.
> >>>>>
> >>>>> 6. partial meta 1 (capture video device): return the AF statistics.
> >>>>>
> >>>>> 7. partial meta 2 (capture video device): return the local contrast
> >>>>>    enhanced statistics.
> >>>>>
> >>>>> 8. partial meta 3 (capture video device): return the local motion
> >>>>>    vector statistics.
> >>>>>
> >>>>> The overall patches of the series is:
> >>>>>
> >>>>> * Patch 1 & 2 are dt-bindings & dts information related to ISP P1 driver.
> >>>>> * Patch 3 adds new timestamp type for Camera AR (Augmented Reality) application
> >>>>> * Patch 4 extends the original V4L2 image & meta formats for ISP P1 driver.
> >>>>> * Patch 5 is the heart of ISP P1 driver. It handles the ISP  HW configuration.
> >>>>>   Moreover, implement standard V4L2 video driver that utilizes
> >>>>>   V4L2 and media framework APIs. Communicate with co-process via SCP
> >>>>>   communication to compose ISP registers in the firmware.
> >>>>>
> >>>>> Here is ISP P1 media topology:
> >>>>> It is included the main/sub sensor, sen-inf sub-devices and len device
> >>>>> which are implemented in below patch[1][2][3][4]:
> >>>>
> >>>> I would be nice if you could provide a branch with those applied.
> >>>>
> >>>
> >>> We apply those patches in the chromeos-4.19 to test.
> >>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19
> >>>
> >>>
> >>>>>
> >>>>> For Mediatek ISP P1 driver, it also depends on MT8183 SCP[5] & IOMMU[6]
> >>>>> patch sets.
> >>>>>
> >>>>> /usr/bin/media-ctl -p -d /dev/media2
> >>>>>
> >>>>> Media controller API version 4.19.89
> >>>>>
> >>>>> Media device information
> >>>>> ------------------------
> >>>>> driver          mtk-cam-p1
> >>>>> model           mtk-cam-p1
> >>>>> serial
> >>>>> bus info        platform:1a000000.camisp
> >>>>> hw revision     0x0
> >>>>> driver version  4.19.89
> >>>>>
> >>>>> Device topology
> >>>>> - entity 1: mtk-cam-p1 (12 pads, 8 links)
> >>>>
> >>>> If I understand correctly, the hardware supports 3 ISP instances, A, B, and C, and only B is being used.
> >>>> Is this correct?
> >>>>
> >>>> So maybe, rename it to mtk-isp-p1-b, to allow mtk-isp-p1-a and mtk-isp-p1-c to be added in the future.
> >>>>
> >>>
> >>> Currently, we only support single-cam in this SoC with upstream driver.
> >>> It is plan in next Mediatek SoC to support multi-cam capabilities. So
> >>> we'd like to keep the naming to avoid confusion.
> >>
> >> I suppose this new Mediatek SoC would use this same driver?
> >> I'm just thinking about backwards compatibility. When you add support for this other SoC, the topology
> >> naming will be different then, right? (I guess it's ok).
> >>
> >
> > Sorry, my last comment should be corrected.
> > The new Mediatek SoC with new ISP HW version will support multi-cam
> > capabilities with new upstream driver. Due to the new enrich function,
> > it may not reuse current driver.
>
> right, thanks for the clarification.
>
> >
> >>>
> >>>>>             type V4L2 subdev subtype Unknown flags 0
> >>>>>             device node name /dev/v4l-subdev0
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1 meta input":0 []
> >>>>
> >>>> I would prefer the name params, or parameters, since input/output is confusing, since this is a output video node.
> >>>>
> >>>
> >>> Ok, we will revise our naming in next patch.
> >>>
> >>>>> pad1: Source
> >>>>> -> "mtk-cam-p1 main stream":0 [ENABLED,IMMUTABLE]
> >>>>
> >>>> Is there any reason for this link to be IMMUTABLE? Can't a use "mtk-cam-p1 packed out" without configuring "mtk-cam-p1 main stream" ?
> >>>>
> >>>
> >>> Yes, you are right. We will remove IMMUTABLE flag in next patch.
> >>>
> >>>>> pad2: Source
> >>>>> -> "mtk-cam-p1 packed out":0 []
> >>>>
> >>>> Same here, maybe "packed stream" ? Just for curiosity, why is it called packed?
> >>>>
> >>>
> >>> Comparing with V4L2_PIX_FMT_SGBRG8, we packed the color bits without no
> >>> padding in the memory. We may revise the naming in next patch.
> >>>
> >>>>> pad3: Source
> >>>>> -> "mtk-cam-p1 partial meta 0":0 []
> >>>>> pad4: Source
> >>>>> -> "mtk-cam-p1 partial meta 1":0 []
> >>>>> pad5: Source
> >>>>> -> "mtk-cam-p1 partial meta 2":0 []
> >>>>> pad6: Source
> >>>>> -> "mtk-cam-p1 partial meta 3":0 []
> >>>>
> >>>> Shouldn't those links be [ENABLED,IMMUTABLE] ?
> >>>>
> >>>> It would be better to have a more intuitive naming, e.g. "mtk-cam-p1 AE/AWB stats", "mtk-cam-p1 AF stats",
> >>>> "mtk-cam-p1 contrast stats", "mtk-cam-p1 motion stats", what do you think?
> >>>>
> >>>> I also would prefer to remove blank spaces.
> >>>>
> >>>> And maybe the prefix could be mtkisp-p1 ? (just to be similar with rkisp1), but I don't have strong feelings about this.
> >>>>
> >>>
> >>> No, these links are optional to setup for userspace.
> >>
> >> Right, I just saw in the patch that you use links to know which video nodes should participate in the stream,
> >> and you wait for STREAM_ON to be called in all video nodes before actually enabling the stream, correct?
> >>
> >> I'm not sure if using the link state is the best option (please see my comment on 5/5).
> >> Instead of waiting for them to call STREAM_ON, userspace could do a request to enable the stream in all the
> >> interesting nodes at once.
> >>
> >>
> >> Regards,
> >> Helen
> >>
> >
> >
> > According to your suggestion, do you have sample code about "userspace
> > could do a request to enable the stream in all the interesting nodes at
> > once"? If this supports, it is helpful for us to simply our current
> > implementation.
>
> I was checking the request api docs [1] in more details and it seems this isn't possible, since
> "A request must contain at least one buffer, otherwise ENOENT is returned", and STREAMON is not listed
> on MEDIA_IOC_REQUEST_ALLOC doc[2].
>
> [1] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/request-api.html
> [2] https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-ioc-request-alloc.html#media-ioc-request-alloc
>
> What bothers me with the current implementation is that users could correctly configure the topology, but when
> calling VIDIOC_STREAMON in one capture node, if userspace doesn't call VIDIOC_STREAMON in all other capture nodes
> with enabled link, streaming will just "hang" (waiting for other nodes) without providing any feedback
> for userspace about what is wrong.

The problem isn't specific to Request API or ISPs. The same would
happen in case of memory to memory devices, which need streaming on
both OUTPUT and CAPTURE queues to be started.

>
> So I was trying to think about how this could be done differently.
> I wonder it if would make sense to extend the Request API to allow calling VIDIOC_STREAMON to multiple nodes at once.
> If not, I guess we could at least add documentation somewhere explaining that calling VIDIOC_STREAMON in all capture
> nodes with enabled link is required to use this driver. Or/And add a dev_info() to print every time VIDIOC_STREAMON
> is called to inform that streaming is being held because it is waiting for other VIDIOC_STREAMON calls.

I'd envision some API to tell the userspace which nodes are required
for streaming. This could be perhaps inferred from the media
controller topology. Then if there are some nodes not started yet,
poll()/DQBUF could return some error.

>
> Regards,
> Helen
>
>
> >
> >
> > Thanks,
> >
> >
> > Jungo
> >
> >
> >>> For naming part, we will align with driver source codes.
> >>>
> >>>>> pad7: Source
> >>>>> pad8: Source
> >>>>> pad9: Source
> >>>>> pad10: Source
> >>>>
> >>>> Why source pads that are not connected to anything? (I guess I need to check the last patch better).
> >>>>
> >>>
> >>> These pads are just reserved purpose.
> >>> We will plan to remove them in next patch.
> >>>
> >>> Thanks,
> >>>
> >>> Jungo
> >>>
> >>>> Regards,
> >>>> Helen
> >>>>
> >>>>> pad11: Sink
> >>>>> <- "1a040000.seninf":4 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 14: mtk-cam-p1 meta input (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video2
> >>>>> pad0: Source
> >>>>> -> "mtk-cam-p1":0 []
> >>>>>
> >>>>> - entity 20: mtk-cam-p1 main stream (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video3
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":1 [ENABLED,IMMUTABLE]
> >>>>>
> >>>>> - entity 26: mtk-cam-p1 packed out (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video4
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":2 []
> >>>>>
> >>>>> - entity 32: mtk-cam-p1 partial meta 0 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video5
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":3 []
> >>>>>
> >>>>> - entity 38: mtk-cam-p1 partial meta 1 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video6
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":4 []
> >>>>>
> >>>>> - entity 44: mtk-cam-p1 partial meta 2 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video7
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":5 []
> >>>>>
> >>>>> - entity 50: mtk-cam-p1 partial meta 3 (1 pad, 1 link)
> >>>>>              type Node subtype V4L flags 0
> >>>>>              device node name /dev/video8
> >>>>> pad0: Sink
> >>>>> <- "mtk-cam-p1":6 []
> >>>>>
> >>>>> - entity 56: 1a040000.seninf (12 pads, 3 links)
> >>>>>              type V4L2 subdev subtype Unknown flags 0
> >>>>>              device node name /dev/v4l-subdev1
> >>>>> pad0: Sink
> >>>>> [fmt:SGRBG10_1X10/3264x2448 field:none colorspace:srgb]
> >>>>> <- "ov8856 2-0010":0 [ENABLED]
> >>>>> pad1: Sink
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> <- "ov02a10 4-003d":0 []
> >>>>> pad2: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad3: Sink
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad4: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> -> "mtk-cam-p1":11 [ENABLED,IMMUTABLE]
> >>>>> pad5: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad6: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad7: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad8: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad9: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad10: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>> pad11: Source
> >>>>> [fmt:SBGGR10_1X10/1600x1200 field:none colorspace:srgb]
> >>>>>
> >>>>> - entity 69: ov8856 2-0010 (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev2
> >>>>> pad0: Source
> >>>>> [fmt:SBGGR10_1X10/3264x2448 field:none colorspace:unknown ycbcr:709]
> >>>>> -> "1a040000.seninf":0 [ENABLED]
> >>>>>
> >>>>> - entity 73: dw9768 2-000c (0 pad, 0 link)
> >>>>>              type V4L2 subdev subtype Lens flags 0
> >>>>>              device node name /dev/v4l-subdev3
> >>>>>
> >>>>> - entity 74: ov02a10 4-003d (1 pad, 1 link)
> >>>>>              type V4L2 subdev subtype Sensor flags 0
> >>>>>              device node name /dev/v4l-subdev4
> >>>>> pad0: Source
> >>>>> [fmt:SRGGB10_1X10/1600x1200 field:none]
> >>>>> -> "1a040000.seninf":1 []
> >>>>>
> >>>>> ===========
> >>>>> = history =
> >>>>> ===========
> >>>>>
> >>>>> version 6:
> >>>>>  - Add port node description in the dt-binding document and device tree
> >>>>>    by Tomasz Figa.
> >>>>>  - Remove RGB format definitions in pixfmt-rgb.rst for kernel v5.5-rc1
> >>>>>    version.
> >>>>>  - Revise help description for VIDEO_MEDIATEK_ISP_PASS1.
> >>>>>  - Apply SCP v21 change in P1 driver by Pi-Hsun Shih.
> >>>>>  - Correct auto suspend timer value for suspend/resume issue.
> >>>>>  - Increase IPI guard timer to 1 second to avoid false alarm command
> >>>>>    timeout event.
> >>>>>  - Fix KE due to no sen-inf sub-device.
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation.
> >>>>>  - Add rst documents for Mediatek meta formats.
> >>>>>  - New meta buffer structure design & re-factoring.
> >>>>>
> >>>>> version 5:
> >>>>>  - Fixed Rob's comment on dt-binding format
> >>>>>  - Fix Tomasz's comment in mtk_isp_pm_suspend function
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>    and new timestamp type in driver
> >>>>>  - Fix buffer en-queue timing issue in v4
> >>>>>  - Remove default link_notify callback function in mtk_cam_media_ops
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 4:
> >>>>>  - Fix Tomasz's comments which are addressed in MTK ISP P1 driver v3
> >>>>>    patch[4]
> >>>>>  - Fix some Tomasz comments which are addressed in DIP's v2 patch[5]
> >>>>>  - Extend Mediatek proprietary image formats to support bayer order
> >>>>>  - Support V4L2_BUF_FLAG_TSTAMP_SRC_SOE for capture devices
> >>>>>
> >>>>> Todo:
> >>>>>  - vb2_ops's buf_request_complete callback function implementation
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>  - Support V4L2_BUF_FLAG_TIMESTAMP_BOOTTIME timestamp flag
> >>>>>  - Align and pack IPI command structures for EC ROM size shrink
> >>>>>
> >>>>> version 3:
> >>>>>  - Remove ISP Pass 1 reserved memory device node and change to use SCP's
> >>>>>    reserved memory region. (Rob Herring)
> >>>>>  - Fix comments of ISP Pass 1 device node & dt-bindings document (Rob Herring)
> >>>>>  - Revise ISP Pass1 Kconfig
> >>>>>  - Add rst documents for Mediatek image formats (Hans Verkuil)
> >>>>>  - Fix kernel warning messages when running v4l2_compliance test
> >>>>>  - Move AFO buffer enqueue & de-queue from request API to non-request
> >>>>>  - mtk_cam-ctrl.h/mtk_cam-ctrl.c
> >>>>>    Revise Mediatek ISP Pass1 specific V4L2 control naming & file licence
> >>>>>    declaration (Hans Verkuil)
> >>>>>    Split GET_BIN_INFO control into two controls to get width & height
> >>>>>    in-dependently (Hans Verkuil)
> >>>>>  - mtk_cam-v4l2-util.h/mtk_cam-v4l2-util.c
> >>>>>    Merging mtk_cam-dev.c and mtk_cam-v4l2-util.c. (Drew Davenport)
> >>>>>    Remove the pix_mode argument in related functions and unreachable code. (Drew Davenport)
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Fix Drew's comments which are addressed in v2 patch
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Refactoring mtk_isp_config & mtk_isp_req_enqueue functions
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Move function declarations from mtk_cam.h to mtk_cam-scp.h (Drew Davenport)
> >>>>>    Fix some Tomasz comments which are addressed in DIP's v1 patch[3]
> >>>>>    Fix ISP de-initialize timing KE issue
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Get the reserved shared memory via SCP driver (Tomasz Figa)
> >>>>>
> >>>>> Todo:
> >>>>>  - Add rst documents for Mediatek meta formats
> >>>>>  - New meta buffer structure design & re-factoring
> >>>>>
> >>>>> version 2:
> >>>>>  - Add 3A enhancement feature which includes:
> >>>>>    Separates 3A pipeline out of frame basis to improve
> >>>>>    AE/AWB (exposure and white balance) performance.
> >>>>>    Add 2 SCP sub-commands for 3A meta buffers.
> >>>>>  - Add new child device to manage P1 shared memory between P1 HW unit
> >>>>>    and co-processor.
> >>>>>  - Remove mediatek,cam_smem.txt & cam_smem dts node in mt8183.dtsi.
> >>>>>  - Revised document wording for dt-bindings documents & dts information.
> >>>>>  - Remove mtk_cam-ctx.h & mtk_cam-dev-ctx-core.c and move these
> >>>>>    source codes to mtk_cam-dev.h & mtk_cam-dev.c.
> >>>>>  - mtk_cam-dev.h / mtk_cam-dev.c
> >>>>>    Revised mtk_cam_video_device & mtk_cam_dev to remove unused structure fields
> >>>>>    or add comments.
> >>>>>    Revised buffer size for LMVO & LCSO.
> >>>>>    Fix pixel format utility function.
> >>>>>    Add vb2_dma_contig_set_max_seg_size to configure DMA max segment size.
> >>>>>  - mtk_cam-v4l2-util.c
> >>>>>    Refactoring V4L2 async mechanism with seninf driver only
> >>>>>    Refactoring CIO (Connection IO) implementation with active sensor
> >>>>>    Revised stream on function for 3A enhancement feature
> >>>>>    Add new V4L2 en-queue/de-queue utility functions for 3A enhancement feature
> >>>>>  - mtk_cam-regs.h / mtk_cam.h / mtk_cam.c
> >>>>>    Add meta buffer index register definitions
> >>>>>    Add meta DMA configuration function.
> >>>>>    Separate with frame-base and non-frame-base en-queue/de-queue functions
> >>>>>    Add isp_setup_scp_rproc function to get RPC handle
> >>>>>    Add mtk_cam_reserved_memory_init for shared memory management
> >>>>>  - mtk_cam-scp.h / mtk_cam-scp.c
> >>>>>    Add new meta strictures for 3A enhancement feature
> >>>>>    Add new IPI command utility function for 3A enhancement feature
> >>>>>    Enhance isp_composer_dma_sg_init function flow
> >>>>>    Shorten overall IPI command structure size
> >>>>>    Remove scp_state state checking
> >>>>>    Improve code readability
> >>>>>  - mtk_cam-smem.h / mtk_cam-smem-dev.c
> >>>>>    Add mtk_cam_alloc_smem_dev to allocate one new child device of ISP driver.
> >>>>>    Handling P1 driver 's reserved memory & allocate DMA buffers based on this
> >>>>>    memory region.
> >>>>>
> >>>>> TODOs:
> >>>>>  - 3A enhancement feature bug fixing
> >>>>>
> >>>>> version 1:
> >>>>>  - Revised driver sources based on Tomasz's comments including
> >>>>>    part1/2/3/4 in RFC V0 patch.
> >>>>>  - Remove DMA cache mechanism.
> >>>>>    Support two new video devices (LCSO/LMVO) for advance camera
> >>>>>    features.
> >>>>>  - Fixed v4l2-compliance test failure items.
> >>>>>  - Add private controls for Mediatek camera middle-ware.
> >>>>>  - Replace VPU driver's APIs with new SCP driver interface for
> >>>>>    co-processor communication.
> >>>>>  - Refactoring mtk_cam_scp.c to use ring-buffers mechanism for IPI
> >>>>>    commands RX handling.
> >>>>>  - Fix internal bugs.
> >>>>>
> >>>>> TODOs:
> >>>>>  - Remove mtk_cam_smem_drv.c & mtk_cam_smem.h and implement DMA pool
> >>>>>    for shared memory management.
> >>>>>  - Revised file names.
> >>>>>  - Support non frame-sync AFO/AAO DMA buffers
> >>>>>
> >>>>> version 0:
> >>>>> - Initial submission
> >>>>>
> >>>>> ==================
> >>>>>  Dependent patch set
> >>>>> ==================
> >>>>>
> >>>>> Camera ISP P1 driver depends on seninf driver, SCP driver.
> >>>>> The patches are listed as following:
> >>>>>
> >>>>> [1]. media: support Mediatek sensor interface driver
> >>>>> https://patchwork.kernel.org/cover/11145845/
> >>>>>
> >>>>> [2]. media: ov8856: Add YAML binding and sensor mode support
> >>>>> https://patchwork.kernel.org/cover/11220785/
> >>>>>
> >>>>> [3]. media: i2c: Add support for OV02A10 sensor
> >>>>> https://patchwork.kernel.org/cover/11284779/
> >>>>>
> >>>>> [4]. media: i2c: add support for DW9768 VCM driver
> >>>>> https://patchwork.kernel.org/cover/11132299/
> >>>>>
> >>>>> [5]. Add support for mt8183 SCP
> >>>>> https://patchwork.kernel.org/cover/11239065/
> >>>>>
> >>>>> [6]. MT8183 IOMMU SUPPORT
> >>>>> https://patchwork.kernel.org/cover/11112765/
> >>>>>
> >>>>> ==================
> >>>>>  Compliance test
> >>>>> ==================
> >>>>>
> >>>>> The v4l2-compliance is built with the below lastest patch.
> >>>>> https://git.linuxtv.org/v4l-utils.git/commit/?id=e9a7593ec6ae98704ecb35ea64948d34c23a5158
> >>>>>
> >>>>> Note 1.
> >>>>> This testing depends on the above seninf, sensors and len patches[1][2][3][4].
> >>>>>
> >>>>> Note 2.
> >>>>> For failed test csaes in video2~8, it is caused by new V4L2 timestamp
> >>>>> called V4L2_BUF_FLAG_TIMESTAMP_BOOTIME.
> >>>>>
> >>>>> Note 3.
> >>>>> The current some failure items are related to Mediatek sensors/len driver [2][3][3]
> >>>>>
> >>>>> /usr/bin/v4l2-compliance -m /dev/media2
> >>>>>
> >>>>> v4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media1 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 11 Interfaces: 11 Pads: 33 Links: 21
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media1: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video25:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.67
> >>>>> Capabilities     : 0x8c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.67
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.67
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video25 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composiv4l2-compliance SHA: not available, 32 bits
> >>>>>
> >>>>> Compliance test for mtk-cam-p1 device /dev/media2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/media2 open: OK
> >>>>> test MEDIA_IOC_DEVICE_INFO: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Media Controller ioctls:
> >>>>> test MEDIA_IOC_G_TOPOLOGY: OK
> >>>>> Entities: 12 Interfaces: 12 Pads: 33 Links: 22
> >>>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> >>>>> test MEDIA_IOC_SETUP_LINK: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/media2: 7, Succeeded: 7, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video2:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x8c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x0c200000
> >>>>> Metadata Output
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000010
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000000e (14)
> >>>>> Name             : mtk-cam-p1 meta input
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100000f   : 0: Source
> >>>>>   Link 0x02000012: to remote pad 0x1000002 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video2 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video2: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video3:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000016
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000014 (20)
> >>>>> Name             : mtk-cam-p1 main stream
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000015   : 0: Sink
> >>>>>   Link 0x02000018: from remote pad 0x1000003 of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video3 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video3: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video4:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04201000
> >>>>> Video Capture Multiplanar
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300001c
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000001a (26)
> >>>>> Name             : mtk-cam-p1 packed out
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100001b   : 0: Sink
> >>>>>   Link 0x0200001e: from remote pad 0x1000004 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video4 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video4: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video5:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000022
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000020 (32)
> >>>>> Name             : mtk-cam-p1 partial meta 0
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000021   : 0: Sink
> >>>>>   Link 0x02000024: from remote pad 0x1000005 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video5 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video5: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video6:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000028
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000026 (38)
> >>>>> Name             : mtk-cam-p1 partial meta 1
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000027   : 0: Sink
> >>>>>   Link 0x0200002a: from remote pad 0x1000006 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video6 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video6: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video7:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x0300002e
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x0000002c (44)
> >>>>> Name             : mtk-cam-p1 partial meta 2
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x0100002d   : 0: Sink
> >>>>>   Link 0x02000030: from remote pad 0x1000007 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video7 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video7: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/video8:
> >>>>>
> >>>>> Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Card type        : mtk-cam-p1
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Driver version   : 4.19.89
> >>>>> Capabilities     : 0x84a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Device Capabilities
> >>>>> Device Caps      : 0x04a00000
> >>>>> Metadata Capture
> >>>>> Streaming
> >>>>> Extended Pix Format
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000034
> >>>>> Type             : V4L Video
> >>>>> Entity Info:
> >>>>> ID               : 0x00000032 (50)
> >>>>> Name             : mtk-cam-p1 partial meta 3
> >>>>> Function         : V4L2 I/O
> >>>>> Pad 0x01000033   : 0: Sink
> >>>>>   Link 0x02000036: from remote pad 0x1000008 of entity 'mtk-cam-p1': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/video8 open: OK
> >>>>> test VIDIOC_QUERYCAP: OK
> >>>>> test VIDIOC_G/S_PRIORITY: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK
> >>>>> test VIDIOC_TRY_FMT: OK
> >>>>> test VIDIOC_S_FMT: OK
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> fail: v4l2-test-buffers.cpp(370): timestamp != V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC && timestamp != V4L2_BUF_FLAG_TIMESTAMP_COPY
> >>>>> fail: v4l2-test-buffers.cpp(486): buf.check(Unqueued, i)
> >>>>> fail: v4l2-test-buffers.cpp(615): testQueryBuf(node, i, q.g_buffers())
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> >>>>> fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
> >>>>> fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/video8: 45, Succeeded: 44, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev0:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000050
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000001 (1)
> >>>>> Name             : mtk-cam-p1
> >>>>> Function         : Video Pixel Formatter
> >>>>> Pad 0x01000002   : 0: Sink
> >>>>>   Link 0x02000012: from remote pad 0x100000f of entity 'mtk-cam-p1 meta input': Data
> >>>>> Pad 0x01000003   : 1: Source
> >>>>>   Link 0x02000018: to remote pad 0x1000015 of entity 'mtk-cam-p1 main stream': Data, Enabled, Immutable
> >>>>> Pad 0x01000004   : 2: Source
> >>>>>   Link 0x0200001e: to remote pad 0x100001b of entity 'mtk-cam-p1 packed out': Data
> >>>>> Pad 0x01000005   : 3: Source
> >>>>>   Link 0x02000024: to remote pad 0x1000021 of entity 'mtk-cam-p1 partial meta 0': Data
> >>>>> Pad 0x01000006   : 4: Source
> >>>>>   Link 0x0200002a: to remote pad 0x1000027 of entity 'mtk-cam-p1 partial meta 1': Data
> >>>>> Pad 0x01000007   : 5: Source
> >>>>>   Link 0x02000030: to remote pad 0x100002d of entity 'mtk-cam-p1 partial meta 2': Data
> >>>>> Pad 0x01000008   : 6: Source
> >>>>>   Link 0x02000036: to remote pad 0x1000033 of entity 'mtk-cam-p1 partial meta 3': Data
> >>>>> Pad 0x01000009   : 7: Source
> >>>>> Pad 0x0100000a   : 8: Source
> >>>>> Pad 0x0100000b   : 9: Source
> >>>>> Pad 0x0100000c   : 10: Source
> >>>>> Pad 0x0100000d   : 11: Sink
> >>>>>   Link 0x0200004e: from remote pad 0x100003d of entity '1a040000.seninf': Data, Enabled, Immutable
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev0 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 0 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev0: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev1:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000052
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000038 (56)
> >>>>> Name             : 1a040000.seninf
> >>>>> Function         : Video Interface Bridge
> >>>>> Pad 0x01000039   : 0: Sink
> >>>>>   Link 0x02000047: from remote pad 0x1000046 of entity 'ov8856 2-0010': Data, Enabled
> >>>>> Pad 0x0100003a   : 1: Sink
> >>>>>   Link 0x0200004c: from remote pad 0x100004b of entity 'ov02a10 4-003d': Data
> >>>>> Pad 0x0100003b   : 2: Sink
> >>>>> Pad 0x0100003c   : 3: Sink
> >>>>> Pad 0x0100003d   : 4: Source
> >>>>>   Link 0x0200004e: to remote pad 0x100000d of entity 'mtk-cam-p1': Data, Enabled, Immutable
> >>>>> Pad 0x0100003e   : 5: Source
> >>>>> Pad 0x0100003f   : 6: Source
> >>>>> Pad 0x01000040   : 7: Source
> >>>>> Pad 0x01000041   : 8: Source
> >>>>> Pad 0x01000042   : 9: Source
> >>>>> Pad 0x01000043   : 10: Source
> >>>>> Pad 0x01000044   : 11: Source
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev1 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 1):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 2):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Sink Pad 3):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 4):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 5):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 6):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 7):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 8):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 9):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 10):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 11):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev1: 125, Succeeded: 125, Failed: 0, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev2:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000054
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000045 (69)
> >>>>> Name             : ov8856 2-0010
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x01000046   : 0: Source
> >>>>>   Link 0x02000047: to remote pad 0x1000039 of entity '1a040000.seninf': Data, Enabled
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev2 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
> >>>>> fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> fail: v4l2-test-subdevs.cpp(147): doioctl(node, VIDIOC_SUBDEV_ENUM_FRAME_SIZE, &fse)
> >>>>> fail: v4l2-test-subdevs.cpp(248): ret && ret != ENOTTY
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: FAIL
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 11 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev2: 48, Succeeded: 44, Failed: 4, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev3:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000056
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x00000049 (73)
> >>>>> Name             : dw9768 2-000c
> >>>>> Function         : Lens Controller
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev3 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> test VIDIOC_G/S_CTRL: OK
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'Camera Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 2 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev3: 41, Succeeded: 40, Failed: 1, Warnings: 0
> >>>>> --------------------------------------------------------------------------------
> >>>>> Compliance test for mtk-cam-p1 device /dev/v4l-subdev4:
> >>>>>
> >>>>> Media Driver Info:
> >>>>> Driver name      : mtk-cam-p1
> >>>>> Model            : mtk-cam-p1
> >>>>> Serial           :
> >>>>> Bus info         : platform:1a000000.camisp
> >>>>> Media version    : 4.19.89
> >>>>> Hardware revision: 0x00000000 (0)
> >>>>> Driver version   : 4.19.89
> >>>>> Interface Info:
> >>>>> ID               : 0x03000058
> >>>>> Type             : V4L Sub-Device
> >>>>> Entity Info:
> >>>>> ID               : 0x0000004a (74)
> >>>>> Name             : ov02a10 4-003d
> >>>>> Function         : Camera Sensor
> >>>>> Pad 0x0100004b   : 0: Source
> >>>>>   Link 0x0200004c: to remote pad 0x100003a of entity '1a040000.seninf': Data
> >>>>>
> >>>>> Required ioctls:
> >>>>> test MC information (see 'Media Driver Info' above): OK
> >>>>>
> >>>>> Allow for multiple opens:
> >>>>> test second /dev/v4l-subdev4 open: OK
> >>>>> test for unlimited opens: OK
> >>>>>
> >>>>> Debug ioctls:
> >>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>>
> >>>>> Input ioctls:
> >>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
> >>>>>
> >>>>> Output ioctls:
> >>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>>
> >>>>> Input/Output configuration ioctls:
> >>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> >>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>>
> >>>>> Sub-Device ioctls (Source Pad 0):
> >>>>> test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_FMT: OK
> >>>>> test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> >>>>> test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> >>>>>
> >>>>> Control ioctls:
> >>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> >>>>> test VIDIOC_QUERYCTRL: OK
> >>>>> fail: v4l2-test-controls.cpp(362): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(431): invalid control 009e0902
> >>>>> test VIDIOC_G/S_CTRL: FAIL
> >>>>> fail: v4l2-test-controls.cpp(549): returned control value out of range
> >>>>> fail: v4l2-test-controls.cpp(665): invalid control 009e0902
> >>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> >>>>> fail: v4l2-test-controls.cpp(830): subscribe event for control 'User Controls' failed
> >>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
> >>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>>> Standard Controls: 10 Private Controls: 0
> >>>>>
> >>>>> Format ioctls:
> >>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> >>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>>> test VIDIOC_G_FMT: OK (Not Supported)
> >>>>> test VIDIOC_TRY_FMT: OK (Not Supported)
> >>>>> test VIDIOC_S_FMT: OK (Not Supported)
> >>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>>> test Cropping: OK (Not Supported)
> >>>>> test Composing: OK (Not Supported)
> >>>>> test Scaling: OK (Not Supported)
> >>>>>
> >>>>> Codec ioctls:
> >>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>>
> >>>>> Buffer ioctls:
> >>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> >>>>> test VIDIOC_EXPBUF: OK (Not Supported)
> >>>>> test Requests: OK (Not Supported)
> >>>>>
> >>>>> Total for mtk-cam-p1 device /dev/v4l-subdev4: 48, Succeeded: 45, Failed: 3, Warnings: 0
> >>>>>
> >>>>> Grand Total for mtk-cam-p1 device /dev/media2: 709, Succeeded: 694, Failed: 15, Warnings: 0
> >>>>>
> >>>>>
> >>>>> Jungo Lin (5):
> >>>>>   media: dt-bindings: mt8183: Added camera ISP Pass 1
> >>>>>   dts: arm64: mt8183: Add ISP Pass 1 nodes
> >>>>>   media: videodev2.h: Add new boottime timestamp type
> >>>>>   media: platform: Add Mediatek ISP P1 image & meta formats
> >>>>>   media: platform: Add Mediatek ISP P1 V4L2 device driver
> >>>>>
> >>>>>  .../bindings/media/mediatek,camisp.txt        |   83 +
> >>>>>  Documentation/media/uapi/v4l/buffer.rst       |   11 +-
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10.rst   |   65 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst  |   90 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12.rst   |   61 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14.rst   |   73 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst  |  110 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8.rst    |   51 +
> >>>>>  .../media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst   |   78 +
> >>>>>  arch/arm64/boot/dts/mediatek/mt8183.dtsi      |   38 +
> >>>>>  drivers/media/platform/Kconfig                |    1 +
> >>>>>  drivers/media/platform/Makefile               |    1 +
> >>>>>  drivers/media/platform/mtk-isp/Kconfig        |   20 +
> >>>>>  .../media/platform/mtk-isp/isp_50/Makefile    |    3 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/Makefile      |    6 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.c  |  636 +++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-hw.h  |   64 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h |  222 ++
> >>>>>  .../mtk-isp/isp_50/cam/mtk_cam-regs.h         |   95 +
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.c     | 2087 +++++++++++++++++
> >>>>>  .../platform/mtk-isp/isp_50/cam/mtk_cam.h     |  244 ++
> >>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   37 +
> >>>>>  include/uapi/linux/videodev2.h                |   41 +
> >>>>>  24 files changed, 4226 insertions(+), 1 deletion(-)
> >>>>>  create mode 100644 Documentation/devicetree/bindings/media/mediatek,camisp.txt
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr10f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr12f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr14f.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8.rst
> >>>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-mtisp-sbggr8f.rst
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/Makefile
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-hw.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-ipi.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam-regs.h
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.c
> >>>>>  create mode 100644 drivers/media/platform/mtk-isp/isp_50/cam/mtk_cam.h
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> Linux-mediatek mailing list
> >>>> Linux-mediatek@lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/linux-mediatek
> >
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 388+ messages in thread

end of thread, other threads:[~2020-05-05 16:26 UTC | newest]

Thread overview: 388+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <Jungo Lin <jungo.lin@mediatek.com>
2019-04-02 10:04 ` [PATCH v1] media: media_device_enum_links32: fix missing reserved field copy Jungo Lin
2019-04-02 10:04   ` Jungo Lin
2019-04-02 10:04   ` Jungo Lin
2019-04-02 11:33   ` Laurent Pinchart
2019-04-02 11:33     ` Laurent Pinchart
2019-04-02 11:33     ` Laurent Pinchart
2019-04-03  0:30     ` Jungo Lin
2019-04-03  0:30       ` Jungo Lin
2019-04-03  0:30       ` Jungo Lin
2019-04-03  1:44 ` [PATCH] media: media_device_enum_links32: clean a reserved field Jungo Lin
2019-04-03  1:44   ` Jungo Lin
2019-04-03  1:44   ` Jungo Lin
2019-05-10  1:57 ` [RFC, V2, 00/11] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` [RFC,V2,00/11] " Jungo Lin
2019-05-10  1:57 ` [RFC, V2, 01/11] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` [RFC,V2,01/11] " Jungo Lin
2019-05-14 19:50   ` Rob Herring
2019-05-14 19:50     ` Rob Herring
2019-05-14 19:50     ` Rob Herring
2019-05-15 13:02     ` Jungo Lin
2019-05-15 13:02       ` Jungo Lin
2019-05-15 13:02       ` Jungo Lin
2019-05-10  1:57 ` [RFC,V2,02/11] dts: arm64: mt8183: Add ISP Pass 1 shared memory node Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57 ` [RFC,V2,03/11] dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-14 19:54   ` Rob Herring
2019-05-14 19:54     ` Rob Herring
2019-05-14 19:54     ` Rob Herring
2019-05-16  6:12     ` Jungo Lin
2019-05-16  6:12       ` Jungo Lin
2019-05-16  6:12       ` Jungo Lin
2019-05-10  1:57 ` [RFC,V2,04/11] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57 ` [RFC, V2, 05/11] media: platform: Add Mediatek ISP Pass 1 driver Kconfig Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` [RFC,V2,05/11] " Jungo Lin
2019-05-10  1:57 ` [RFC, V2, 06/11] media: platform: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-05-10  1:57   ` Jungo Lin
2019-05-10  1:57   ` [RFC,V2,06/11] " Jungo Lin
2019-05-13  8:35   ` [RFC, V2, 06/11] " Hans Verkuil
2019-05-13  8:35     ` Hans Verkuil
2019-05-13  8:35     ` [RFC,V2,06/11] " Hans Verkuil
2019-05-15 12:49     ` [RFC, V2, 06/11] " Jungo Lin
2019-05-15 12:49       ` Jungo Lin
2019-05-15 12:49       ` Jungo Lin
2019-05-10  1:58 ` [RFC,V2,07/11] media: platform: Add Mediatek ISP P1 private control Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-13  8:46   ` Hans Verkuil
2019-05-13  8:46     ` Hans Verkuil
2019-05-13  8:46     ` Hans Verkuil
     [not found]     ` <49a8ba54-aba4-1915-6732-987a58e8bd3c-qWit8jRvyhVmR6Xm/wNWPw@public.gmane.org>
2019-05-14  6:23       ` Jungo Lin
2019-05-14  6:23         ` Jungo Lin
2019-05-14  6:23         ` Jungo Lin
2019-10-02 10:55     ` Sakari Ailus
2019-10-02 10:55       ` Sakari Ailus
2019-10-02 10:55       ` Sakari Ailus
2019-10-02 11:02       ` Sakari Ailus
2019-10-02 11:02         ` Sakari Ailus
2019-10-02 11:02         ` Sakari Ailus
     [not found] ` <Jungo Lin <jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
2019-05-10  1:58   ` [RFC,V2,08/11] media: platform: Add Mediatek ISP P1 V4L2 functions Jungo Lin
2019-05-10  1:58     ` Jungo Lin
2019-05-10  1:58     ` Jungo Lin
2019-05-24 18:49     ` Drew Davenport
2019-05-24 18:49       ` Drew Davenport
2019-05-24 18:49       ` Drew Davenport
2019-05-28  1:00       ` Jungo Lin
2019-05-28  1:00         ` Jungo Lin
2019-05-28  1:00         ` Jungo Lin
2019-05-10  1:58 ` [RFC,V2,09/11] media: platform: Add Mediatek ISP P1 device driver Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-24 21:19   ` [RFC, V2, 09/11] " Drew Davenport
2019-05-24 21:19     ` Drew Davenport
2019-05-24 21:19     ` [RFC,V2,09/11] " Drew Davenport
2019-05-27 13:07     ` [RFC, V2, 09/11] " Jungo Lin
2019-05-27 13:07       ` Jungo Lin
2019-05-27 13:07       ` Jungo Lin
2019-05-10  1:58 ` [RFC, V2, 10/11] media: platform: Add Mediatek ISP P1 SCP communication Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-10  1:58   ` [RFC,V2,10/11] " Jungo Lin
2019-05-10  1:58 ` [RFC, V2, 11/11] media: platform: Add Mediatek ISP P1 shared memory device Jungo Lin
2019-05-10  1:58   ` Jungo Lin
2019-05-10  1:58   ` [RFC,V2,11/11] " Jungo Lin
2019-08-07 12:47 ` [RFC, v4, 0/4] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Jungo Lin
2019-08-07 12:47   ` Jungo Lin
2019-08-07 12:47   ` [RFC,v4,0/4] " Jungo Lin
2019-08-07 12:48   ` [RFC,v4,1/4] media: dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-21 19:47     ` Rob Herring
2019-08-21 19:47       ` Rob Herring
2019-08-21 19:47       ` Rob Herring
2019-08-22 12:47       ` Jungo Lin
2019-08-22 12:47         ` Jungo Lin
2019-08-22 12:47         ` Jungo Lin
     [not found]     ` <20190807124803.29884-2-jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
2019-08-21 20:17       ` Rob Herring
2019-08-21 20:17         ` Rob Herring
2019-08-21 20:17         ` Rob Herring
2019-08-22 12:48         ` Jungo Lin
2019-08-22 12:48           ` Jungo Lin
2019-08-22 12:48           ` Jungo Lin
2019-08-07 12:48   ` [RFC,v4,2/4] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-07 12:48   ` [RFC, v4, 3/4] media: platform: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-07 12:48     ` [RFC,v4,3/4] " Jungo Lin
2019-08-07 12:48   ` [RFC,v4,4/4] media: platform: Add Mediatek ISP P1 V4L2 device driver Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-08-07 12:48     ` Jungo Lin
2019-09-02  7:51 ` [RFC, v5, 0/5] media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Jungo Lin
2019-09-02  7:51   ` Jungo Lin
2019-09-02  7:51   ` [RFC,v5,0/5] " Jungo Lin
2019-09-02  7:51   ` [RFC,v5, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02 15:17     ` [RFC, v5, " Rob Herring
2019-09-02 15:17       ` Rob Herring
2019-09-02 15:17       ` [RFC,v5, " Rob Herring
2019-09-02  7:51   ` [RFC,v5, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51   ` [RFC,v5, 3/5] media: videodev2.h: Add new boottime timestamp type Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51   ` [RFC, v5, 4/5] media: pixfmt: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51     ` [RFC,v5, " Jungo Lin
2019-09-02  7:51   ` [RFC, v5, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver Jungo Lin
2019-09-02  7:51     ` Jungo Lin
2019-09-02  7:51     ` [RFC,v5, " Jungo Lin
2019-12-19  5:49 ` [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Jungo Lin
2019-12-19  5:49   ` Jungo Lin
2019-12-19  5:49   ` [v6, 1/5] media: dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2020-03-31 15:34     ` Helen Koike
2020-03-31 15:34       ` Helen Koike
2020-03-31 15:34       ` Helen Koike
2020-04-10 10:04       ` Jungo Lin
2020-04-10 10:04         ` Jungo Lin
2020-04-10 10:04         ` Jungo Lin
2019-12-19  5:49   ` [v6, 2/5] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2019-12-19  5:49   ` [v6, 3/5] media: videodev2.h: Add new boottime timestamp type Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2020-01-07 14:10     ` Hans Verkuil
2020-01-07 14:10       ` Hans Verkuil
2020-01-07 14:10       ` Hans Verkuil
     [not found]       ` <e833b88ba74945c495a102c98cd54725@mtkmbs07n1.mediatek.inc>
2020-01-10  9:59         ` Jungo Lin
2020-01-10 10:08       ` Jungo Lin
2020-01-10 10:08         ` Jungo Lin
2020-01-10 10:08         ` Jungo Lin
2019-12-19  5:49   ` [v6, 4/5] media: platform: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2020-04-03  2:30     ` Laurent Pinchart
2020-04-03  2:30       ` Laurent Pinchart
2020-04-03  2:30       ` Laurent Pinchart
2020-04-10 10:00       ` Jungo Lin
2020-04-10 10:00         ` Jungo Lin
2020-04-10 10:00         ` Jungo Lin
2019-12-19  5:49   ` [v6, 5/5] media: platform: Add Mediatek ISP P1 V4L2 device driver Jungo Lin
2019-12-19  5:49     ` Jungo Lin
2020-01-23 13:59     ` Hans Verkuil
2020-01-23 13:59       ` Hans Verkuil
2020-01-23 13:59       ` Hans Verkuil
2020-01-28  2:13       ` Jungo Lin
2020-01-28  2:13         ` Jungo Lin
2020-01-28  2:13         ` Jungo Lin
2020-03-31 15:34     ` Helen Koike
2020-03-31 15:34       ` Helen Koike
2020-04-09  2:05       ` Jungo Lin
2020-04-09  2:05         ` Jungo Lin
2020-04-14 12:25         ` Helen Koike
2020-04-14 12:25           ` Helen Koike
     [not found]           ` <b2c30e560e9b4ec488957ca62bae09fe@mtkmbs01n2.mediatek.inc>
2020-05-04 12:27             ` Jungo Lin
2020-05-04 12:27               ` Jungo Lin
2020-05-04 12:27               ` Jungo Lin
2020-05-05 15:38               ` Helen Koike
2020-05-05 15:38                 ` Helen Koike
2020-05-05 15:38                 ` Helen Koike
2020-04-02 16:45     ` Dafna Hirschfeld
2020-04-02 16:45       ` Dafna Hirschfeld
2020-04-09  2:49       ` Jungo Lin
2020-04-09  2:49         ` Jungo Lin
2020-03-31 15:34   ` [v6, 0/5] media: media: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Helen Koike
2020-03-31 15:34     ` Helen Koike
2020-03-31 15:34     ` Helen Koike
2020-04-10 10:32     ` Jungo Lin
2020-04-10 10:32       ` Jungo Lin
2020-04-14 12:25       ` Helen Koike
2020-04-14 12:25         ` Helen Koike
2020-04-14 12:25         ` Helen Koike
     [not found]         ` <1fd3615eb18f48ada186bfe228fc907b@mtkmbs01n2.mediatek.inc>
2020-05-04 12:40           ` Jungo Lin
2020-05-04 12:40             ` Jungo Lin
2020-05-05 15:30             ` Helen Koike
2020-05-05 15:30               ` Helen Koike
2020-05-05 15:30               ` Helen Koike
2020-05-05 16:18               ` Tomasz Figa
2020-05-05 16:18                 ` Tomasz Figa
2020-05-05 16:18                 ` Tomasz Figa
     [not found] <jungo.lin@mediatek.com>
2019-06-11  3:53 ` [RFC, V3 0/9] " Jungo Lin
2019-06-11  3:53   ` Jungo Lin
2019-06-11  3:53   ` [RFC,V3 " Jungo Lin
2019-06-11  3:53   ` [RFC,v3 1/9] dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53   ` [RFC,v3 2/9] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53   ` [RFC,v3 3/9] media: platform: Add Mediatek ISP Pass 1 driver Kconfig Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53   ` [RFC, v3 4/9] media: platform: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` [RFC,v3 " Jungo Lin
2019-06-11  3:53   ` [RFC,v3 5/9] media: platform: Add Mediatek ISP P1 V4L2 control Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-07-01  5:50     ` Tomasz Figa
2019-07-01  5:50       ` Tomasz Figa
2019-07-01  5:50       ` Tomasz Figa
2019-07-02 11:34       ` Jungo Lin
2019-07-02 11:34         ` Jungo Lin
2019-07-02 11:34         ` Jungo Lin
2019-06-11  3:53   ` [RFC,v3 6/9] media: platform: Add Mediatek ISP P1 V4L2 functions Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-07-10  9:54     ` Tomasz Figa
2019-07-10  9:54       ` Tomasz Figa
2019-07-10  9:54       ` Tomasz Figa
2019-07-18  4:39       ` Jungo Lin
2019-07-18  4:39         ` Jungo Lin
2019-07-18  4:39         ` Jungo Lin
2019-07-23 10:21         ` Tomasz Figa
2019-07-23 10:21           ` Tomasz Figa
2019-07-23 10:21           ` Tomasz Figa
2019-07-24  4:31           ` Jungo Lin
2019-07-24  4:31             ` Jungo Lin
2019-07-24  4:31             ` Jungo Lin
2019-07-26  5:49             ` Tomasz Figa
2019-07-26  5:49               ` Tomasz Figa
2019-07-26  5:49               ` Tomasz Figa
2019-07-29  1:18               ` Jungo Lin
2019-07-29  1:18                 ` Jungo Lin
2019-07-29  1:18                 ` Jungo Lin
2019-07-29 10:04                 ` Tomasz Figa
2019-07-29 10:04                   ` Tomasz Figa
2019-07-29 10:04                   ` Tomasz Figa
2019-07-30  1:44                   ` Jungo Lin
2019-07-30  1:44                     ` Jungo Lin
2019-07-30  1:44                     ` Jungo Lin
2019-08-05  9:59                     ` Tomasz Figa
2019-08-05  9:59                       ` Tomasz Figa
2019-08-05  9:59                       ` Tomasz Figa
2019-06-11  3:53   ` [RFC,v3 7/9] media: platform: Add Mediatek ISP P1 device driver Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-07-10  9:56     ` Tomasz Figa
2019-07-10  9:56       ` Tomasz Figa
2019-07-10  9:56       ` Tomasz Figa
2019-07-20  9:58       ` Jungo Lin
2019-07-20  9:58         ` Jungo Lin
2019-07-20  9:58         ` Jungo Lin
2019-07-25  9:23         ` Tomasz Figa
2019-07-25  9:23           ` Tomasz Figa
2019-07-25  9:23           ` Tomasz Figa
2019-07-26  7:23           ` Jungo Lin
2019-07-26  7:23             ` Jungo Lin
2019-07-26  7:23             ` Jungo Lin
2019-08-06  9:47             ` Tomasz Figa
2019-08-06  9:47               ` Tomasz Figa
2019-08-06  9:47               ` Tomasz Figa
2019-08-07  2:11               ` Jungo Lin
2019-08-07  2:11                 ` Jungo Lin
2019-08-07  2:11                 ` Jungo Lin
2019-08-07 13:25                 ` Tomasz Figa
2019-08-07 13:25                   ` Tomasz Figa
2019-08-07 13:25                   ` Tomasz Figa
2019-06-11  3:53   ` [RFC,v3 8/9] media: platform: Add Mediatek ISP P1 SCP communication Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` Jungo Lin
     [not found]     ` <20190611035344.29814-9-jungo.lin-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org>
2019-07-10  9:58       ` Tomasz Figa
2019-07-10  9:58         ` Tomasz Figa
2019-07-10  9:58         ` Tomasz Figa
2019-07-21  2:18         ` Jungo Lin
2019-07-21  2:18           ` Jungo Lin
2019-07-21  2:18           ` Jungo Lin
2019-07-25 10:56           ` [RFC, v3 " Tomasz Figa
2019-07-25 10:56             ` Tomasz Figa
2019-07-25 10:56             ` [RFC,v3 " Tomasz Figa
2019-07-26  8:07             ` Jungo Lin
2019-07-26  8:07               ` Jungo Lin
2019-07-26  8:07               ` Jungo Lin
2019-06-11  3:53   ` [RFC, v3 9/9] media: platform: Add Mediatek ISP P1 shared memory device Jungo Lin
2019-06-11  3:53     ` Jungo Lin
2019-06-11  3:53     ` [RFC,v3 " Jungo Lin
2019-07-01  7:25     ` Tomasz Figa
2019-07-01  7:25       ` Tomasz Figa
2019-07-01  7:25       ` Tomasz Figa
2019-07-05  3:33       ` Jungo Lin
2019-07-05  3:33         ` Jungo Lin
2019-07-05  3:33         ` Jungo Lin
2019-07-05  4:22         ` [RFC, v3 " Tomasz Figa
2019-07-05  4:22           ` Tomasz Figa
2019-07-05  4:22           ` [RFC,v3 " Tomasz Figa
     [not found]           ` <CAAFQd5BaTQ-Q7gsE0X+d4_81OZq9WHaCYkmALt7_4A1JFo=_8g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-07-05  5:44             ` Jungo Lin
2019-07-05  5:44               ` Jungo Lin
2019-07-05  5:44               ` Jungo Lin
2019-07-05  7:59             ` Jungo Lin
2019-07-05  7:59               ` Jungo Lin
2019-07-05  7:59               ` Jungo Lin
2019-07-23  7:20               ` [RFC, v3 " Tomasz Figa
2019-07-23  7:20                 ` Tomasz Figa
2019-07-23  7:20                 ` [RFC,v3 " Tomasz Figa
     [not found]                 ` <CAAFQd5AaNFpMGCVJREY85n8UetEwd99TOka8-ECoLzMbMkos_g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-07-23  8:21                   ` [RFC, v3 " Jungo Lin
2019-07-23  8:21                     ` Jungo Lin
2019-07-23  8:21                     ` Jungo Lin
2019-07-26  5:15                     ` Tomasz Figa
2019-07-26  5:15                       ` Tomasz Figa
2019-07-26  5:15                       ` Tomasz Figa
     [not found]                       ` <CAAFQd5Bh80N+cMhz=eyHUGJLaE5uuypOawQvHrTgGSMDvmcpLA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-07-26  7:41                         ` Christoph Hellwig
2019-07-26  7:41                           ` Christoph Hellwig
2019-07-26  7:41                           ` Christoph Hellwig
2019-07-26  7:42                           ` Tomasz Figa
2019-07-26  7:42                             ` Tomasz Figa
2019-07-26  7:42                             ` Tomasz Figa
     [not found]                             ` <CAAFQd5CXwRm-3jD+rfNNDNLH=gT_i0QYSAG3XBo3SJnPTY56_w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-07-26 11:04                               ` Robin Murphy
2019-07-26 11:04                                 ` Robin Murphy
2019-07-26 11:04                                 ` Robin Murphy
2019-07-26 11:04                                 ` Robin Murphy
     [not found]                                 ` <4460bc91-352a-7f3a-cbed-1b95e743ca8c-5wv7dgnIgG8@public.gmane.org>
2019-07-26 11:59                                   ` Jungo Lin
2019-07-26 11:59                                     ` Jungo Lin
2019-07-26 11:59                                     ` Jungo Lin
2019-07-26 11:59                                     ` Jungo Lin
2019-07-26 14:04                                     ` Tomasz Figa
2019-07-26 14:04                                       ` Tomasz Figa
2019-07-26 14:04                                       ` Tomasz Figa
2019-07-26 14:04                                       ` Tomasz Figa
2019-03-28  9:56 [RFC V1 00/12] meida: platform: mtk-isp: Add Mediatek ISP Pass 1 driver Jungo Lin
2019-03-28  9:56 ` Jungo Lin
2019-03-28  9:56 ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 01/12] dt-bindings: mt8183: Add binding for ISP Pass 1 reserved memory Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 02/12] dts: arm64: mt8183: Add ISP Pass 1 shared memory node Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 03/12] dt-bindings: mt8183: Added cam-smem dt-bindings Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 04/12] dt-bindings: mt8183: Added camera ISP Pass 1 Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 05/12] dts: arm64: mt8183: Add ISP Pass 1 nodes Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 06/12] media: platform: Add Mediatek ISP Pass 1 driver Kconfig Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 07/12] media: platform: Add Mediatek ISP P1 image & meta formats Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 08/12] media: platform: Add Mediatek ISP P1 private control Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 09/12] media: platform: Add Mediatek ISP P1 V4L2 functions Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 10/12] media: platform: Add Mediatek ISP P1 device driver Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 11/12] media: platform: Add Mediatek ISP P1 SCP communication Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56 ` [RFC V1 12/12] media: platform: Add Mediatek ISP P1 shared memory driver Jungo Lin
2019-03-28  9:56   ` Jungo Lin
2019-03-28  9:56   ` Jungo Lin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.